I l@ve RuBoard Next Section

• | Table of Contents
• | Examples
Microsoft SQL Server™ 2000: A Guide to Enhancements and New Features
By Rahul Sharma
Publisher | : Addison Wesley
Pub Date | : March 19, 2002
ISBN | : 0-201-75283-2
Pages | : 592

Microsoft's SQL Server(TM) 2000 promises performance, scalability, and speed. Microsoft SQL Server(TM) 2000: A Guide to Enhancements and New Features shows database administrators and developers how to deliver on these promises. This book illustrates all the significant enhancements and additions in SQL Server 2000 and shows how they can be used most effectively. Readers learn by example how to use security features, replication, backup procedures, and data warehousing and optimization techniques.

Beginning with an overview of SQL Server 2000, this book discusses online transaction processing (OLTP) and online analytical processing (OLAP), features a tour of different SQL Server releases, and offers a guide to installation. The author describes and demonstrates the changes since SQL Server 7.0, thoroughly exploring SQL Server 2000's capacity as a Web-enabled database server. Readers are then immersed in advanced database administration topics such as performance optimization and debugging techniques.

Microsoft SQL Server(TM) 2000 also serves as a resource for those new to SQL Server, providing a basic introduction and details about how to make a conversion work, as well as a feature-by-feature comparison to Oracle 9i. Database administrators will find a collection of clearly explained scripts and utilities to simplify day-to-day tasks. This book concludes with a discussion of how SQL Server fits into the .NET environment and what to expect of the next release.

Sample code is used extensively to illustrate feature sets and illuminate topics covered. The accompanying CD includes this sample code as well as scripts and utilities. Rahul Sharma also shares his tips for optimizing applications' performance with adjustments to code, settings, and hardware. He alerts readers to common errors and fixes, and provides a checklist to guide readers through the installation and verification of the database server's stability. Rahul has translated his experience into knowledge and ready-to-use tools that database administrators and developers need to realize the many benefits of moving to SQL Server 2000.

I l@ve RuBoard Next Section


I l@ve RuBoard

• | Table of Contents
• | Examples
Microsoft SQL Server™ 2000: A Guide to Enhancements and New Features
By Rahul Sharma
Publisher | : Addison Wesley
Pub Date | : March 19, 2002
ISBN | : 0-201-75283-2
Pages | : 592

Copyright
Preface
 | Short Outline of the Book
 | About the Author
Acknowledgments
Chapter 1. SQL Server 2000 Overview
 | SQL Server Editions
 | Hardware Requirements for Installation
 | Installation of SQL Server 2000
 | Multiple Instances
 | Network Protocols
 | Collations Support for Internationalization
 | Checklist for a Successful Install of SQL Server
 | Maintenance and Integrity Checks
 | Summary
Part I. New Features and Improvements over SQL Server 7.0
 | Chapter 2. Enhancements and Changes to Existing Features
 |  | Feature Sets
 |  | Summary
 | Chapter 3. XML Support in SQL Server 2000
 |  | Generating XML with the SELECT Statement
 |  | Generating XML over the Internet
 |  | Retrieving XML-Formatted Data from SQL Server
 |  | Reserved Characters
 |  | XML Templates
 |  | Using EXPLICIT Mode
 |  | XSL Stylesheets
 |  | XML System Stored Procedures
 |  | OPENXML
 |  | Writing Queries Against an XML Document
 |  | Summary
 | Chapter 4. Engine Enhancements in SQL Server 2000
 |  | The Enhancements
 |  | Interacting with Data
 |  | Tables and Indexes
 |  | Logging and Recovery
 |  | Improved Backup Functionality
 |  | Administrative Improvements
 |  | Dynamic Tuning
 |  | Data Storage Components
 |  | Summary
 | Chapter 5. DTS Enhancements
 |  | New Features in DTS
 |  | Using DTS Designer
 |  | Data Pump: Transforming Data
 |  | Options for Saving DTS Packages
 |  | DTS as an Application Development Platform
 |  | Summary
 | Chapter 6. Profiler and Index-Tuning Wizard Enhancements
 |  | SQL Profiler Terminology
 |  | Starting Profiler Trace
 |  | Templates
 |  | Defining Your Own Trace Using Stored Procedures
 |  | Defining a Server-Side Trace
 |  | Known Bug in SQL Server 2000 Profiler
 |  | Index Tuning Wizard
 |  | Analyzing Index Tuning Wizard Output
 |  | Starting the Index Tuning Wizard
 |  | Itwiz.exe
 |  | Summary
 | Chapter 7. Replication Enhancements
 |  | Types of Replication
 |  | Replication Options
 |  | New Features and Improvements
 |  | Replication Usability
 |  | Log Shipping
 |  | Summary
 | Chapter 8. New Data Types
 |  | bigint
 |  | sql_variant
 |  | Table
 |  | Summary
 | Chapter 9. User-Defined Functions
 |  | Types of User-Defined Functions
 |  | Obtaining Information About Functions
 |  | Summary
 | Chapter 10. Indexed Views
 |  | Performance Gains from Indexed Views
 |  | Benefits of Using Indexed Views
 |  | Query Optimizer
 |  | Design Considerations
 |  | Creating an Indexed View
 |  | Maintaining Indexed Views
 |  | Summary
 | Chapter 11. Trigger Alternatives
 |  | Types of Triggers
 |  | Designing INSTEAD OF Triggers
 |  | Summary
 | Chapter 12. Meta Data Services Enhancements
 |  | Meta Data Fundamentals
 |  | Information Model Fundamentals
 |  | New Features in Meta Data Services
 |  | Summary
 | Chapter 13. Tool Enhancements
 |  | Query Analyzer
 |  | Templates
 |  | Copy Database Wizard
 |  | Summary
 | Chapter 14. Backup and Recovery Enhancements
 |  | Backup and Restore Architecture
 |  | Recovery Models
 |  | Switching Recovery Models
 |  | Summary
 | Chapter 15. Analysis Services Enhancements
 |  | Cube Enhancements
 |  | Dimension Enhancements
 |  | Data Mining Enhancements
 |  | Other Enhancements
 |  | Summary
 | Chapter 16. Distributed Partitioned Views
 |  | What Are Distributed Partitioned Views?
 |  | Designing Systems That Maximize Distributed Partitioned View Performance
 |  | Backing Up and Restoring Federated Database Servers
 |  | Updatable Partitioned Views
 |  | Summary
 | Chapter 17. T-SQL Debugger
 |  | Summary
 | Chapter 18. Cascading Declarative Referential Integrity
 |  | Cascading Referential Integrity Constraints
 |  | Multiple Cascading Actions
 |  | Triggers and Cascading Referential Actions
 |  | Cascading Referential Constraints Catalog Information
 |  | Summary
 | Chapter 19. Collation Enhancements
 |  | How Character Data Is Stored
 |  | International Data and Unicode
 |  | Sort Order
 |  | What Is Unicode and How Can It Be Used?
 |  | Data Types in SQL Server 2000
 |  | Collation in SQL Server 2000
 |  | Collations Specified at Multiple Levels
 |  | Collation Precedence
 |  | COLLATE Keyword
 |  | Limitations of the COLLATE Keyword
 |  | Issues with Defining Collation at the Column Level
 |  | LCIDs and Collations
 |  | ISO Strings and Collations
 |  | Multilingual Data in the User Interface
 |  | Multilingual Information in the Grid and SQL Panes of SQL Query Analyzer
 |  | Format Issues in the Query Designer
 |  | Sort Order
 |  | Double-Byte (DBCS) Characters
 |  | Getting to SQL Server Data (Data Access Methods)
 |  | Multilingual Transact-SQL
 |  | Summary
Part II. Advanced DBA Topics
 | Chapter 20. SQL-Distributed Management Objects
 |  | What Is SQL-DMO?
 |  | SQL-DMO Object Model
 |  | ExecuteImmediate Method
 |  | Script Method
 |  | SQL-DMO Code for Executing a Job
 |  | SQL-DMO Code to Play with Triggers
 |  | Summary
 | Chapter 21. Microsoft Desktop Engine
 |  | Portable and Compatible
 |  | Features and Limitations
 |  | MSDE Install
 |  | Scenarios for Using MSDE
 |  | Developing Applications with MSDE
 |  | Summary
 | Chapter 22. Administration
 |  | Database Consistency Checks (DBCCs)
 |  | Information Schema Views
 |  | Orphaned Sessions
 |  | Linked Servers
 |  | Summary
 | Chapter 23. Debugging Database Performance Issues
 |  | Performance Monitor or Task Manager?
 |  | Performance Monitor
 |  | Task Manager
 |  | Performance Monitor Counters
 |  | Settings
 |  | Event Viewer
 |  | SQL Server Profiler
 |  | Hardware Issues
 |  | Database Setup
 |  | SQL Tips
 |  | Summary
 | Chapter 24. SQL Server 2000 Service Pack 1
 |  | Before the Install
 |  | Installing Service Pack 1
 |  | Install Database Components SP1
 |  | Removing Service Pack 1/Rolling Back to the Previous State
 |  | Fixing Bugs
 |  | Summary
Part III. Comparison with Oracle 9i
 | Chapter 25. Oracle 9i vs. SQL Server 2000
 |  | Database Definition
 |  | Database System Catalogs
 |  | Physical and Logical Storage Structures
 |  | Striping Data
 |  | Transaction Logs and Automatic Recovery
 |  | Networks
 |  | Comparison of Features Provided by the Two RDBMS
 |  | New Features in Oracle 9i
 |  | Summary
 | Chapter 26. Migrating from Oracle 9i Databases to SQL Server 2000
 |  | Database Security and Roles
 |  | Defining Database Objects
 |  | Indexed Views/Materialized Views
 |  | Data Types
 |  | Enforcing Data Integrity and Business Rules
 |  | Transactions, Locking, and Concurrency
 |  | Remote Transactions
 |  | Distributed Transactions
 |  | Two-Phase Commit Processing
 |  | SQL Language Support
 |  | Manipulating Data in Identity and Timestamp Columns
 |  | Locking Requested Rows
 |  | Row Aggregates and the Compute Clause
 |  | Reading and Modifying BLOBs
 |  | Functions
 |  | Conditional Tests
 |  | Converting Values to Different Data Types
 |  | User-Defined Functions
 |  | Comparison Operators
 |  | String Concatenation
 |  | Control-of-Flow Language
 |  | Developing and Administering Database Replication
 |  | Moving Data and Applications
 |  | Summary
Part IV. Scripts and Utilities
 | Chapter 27. Administration and Utility Scripts
 |  | Scripts
 |  | DTS Packages
 |  | Copying Databases from Server to Server
 |  | Summary
Part V. SQL Server.NET
 | Chapter 28. SQL Server 2000 and .NET
 |  | What Is Microsoft.NET?
 |  | From Products Vendor to Services Vendor
 |  | Web Services
 |  | .NET Servers
 |  | SQL Server 2000 and .NET
 |  | Next Release of Microsoft Server
 |  | Windows XP and .NET
 |  | Summary
 | Appendix
 |  | SQL Server Resources
 |  | CD-ROM Materials
 |  | MSDE Install Sample VB Application
I l@ve RuBoard


Copyright

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Addison-Wesley was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals.

The author and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.

The publisher offers discounts on this book when ordered in quantity for special sales. For more information, please contact:

Pearson Education Corporate Sales Division

201 W. 103rd Street

Indianapolis, IN 46290

(800) 428-5331

corpsales@pearsoned.com

Visit AW on the Web: www.aw.com/cseng/

Library of Congress Control Number: 2002101261

Copyright © 2002 by Pearson Education, Inc.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher. Printed in the United States of America. Published simultaneously in Canada.

For information on obtaining permission for use of material from this work, please submit a written request to:

Pearson Education, Inc.

Rights and Contracts Department

75 Arlington Street, Suite 300

Boston, MA 02116

Fax: (617) 848-7047

Text printed on recycled paper

1 2 3 4 5 6 7 8 9 10—MA—0605040302

First printing, March 2002


Preface

This book is written for SQL Server developers, DBAs, database managers, systems engineers, and business analysts. It does briefly cover the basics of SQL Server in Part IV, where SQL Server 2000 is compared with Oracle 9i, but throughout the book it is assumed that you have some familiarity with database concepts and SQL Server (any version).

When I began writing this book, I had these goals in mind:

  • Cover the new feature sets of SQL Server 2000 and its enhancements over SQL Server 7.0.

  • Give a checklist of common errors made with SQL Server and point out how to rectify them to make sure that installs and servers are functioning correctly.

  • Provide sample code wherever possible to illustrate the feature sets and emphasize the points.

  • Avoid excessive screenshots and other fillers often used in computer books.

  • Provide tips (on SQL Server code, settings, and hardware) for improving the performance of SQL Server applications.

  • Provide SQL Server developers and DBAs with scripts and utilities that ease their day-to-day administration jobs.

  • Provide a consolidated list comparing features of Oracle 9i and SQL Server 2000 and provide examples of what users should look out for when they start converting their Oracle database to SQL Server.

  • Provide information about SQL Server.NET and how SQL Server 2000 fits into the .NET environment.

You, the readers, will have to judge for yourselves whether I have met these goals have been met or not. Regardless of the degree of success, my hope is that at least the effort that went into writing this book will be evident.

At times where I had to illustrate the command parameters (as for the Index Tuning Wizard—itwiz.exe) and describe what the command line parameters are, it may seem like a rehash of SQL Server Books Online (BOL). This is because the commands are exactly the same, and there can be no better description than BOL for what those command line parameters stand for. They are included only if necessary to illustrate a particular command functionality. All credit for these descriptions of command parameters goes to the BOL team at Microsoft.


Short Outline of the Book

The book is divided into five parts that contain 28 chapters.

Chapter 1: Overview of SQL Server 2000 and what it has to offer in the world of OLTP (online transaction processing) and OLAP (online analytical processing). The SQL Server install, different editions and their feature set differences, upgrading from the previous version to SQL Server 2000 and between different editions, multiple instances, and how to debug SQL Server install are also covered in this part. It also contains a checklist for checking the database server's stability from time to time.

Part I: New Features and Improvements over SQL Server 7.0: Listing and brief description of the enhancements and the new features of SQL Server 2000. Chapters 2–19 cover SQL Server 7.0 (in brief) and what is replaced/added in SQL Server 2000 (in detail, along with examples). Emphasis is on XML and why/how SQL Server 2000 is a Web-enabled database server, how data can be retrieved/added/modified through XML, SQL 2000's tight integration with the Internet, and other new features such as indexed views, DPVs (distributed partitioned views), UDFs (user-defined functions), debugger, and so on.

Part II: Advanced DBA Topics: Chapters 20–24 cover some advanced DBA topics such as using the object models of SQL-DMO to write effective programs, DBCC (database consistency checks) commands and performance optimization and debugging techniques. Orphaned sessions and MSDE are discussed. The installation and bug fixes in SQL Server 2000 Service Pack 1 are also discussed.

Part III: Comparison with Oracle 9i: Since Oracle is the biggest competitor of SQL Server 2000 and since many companies that use Oracle are either shifting to SQL Server 2000 (because it's economically viable) or are planning to support SQL Server 2000 as well in order to broaden their customer base, it is necessary to know the differences between the two and how one can convert applications from Oracle to SQL Server and what it takes from the database perspective to do so.

SQL Server basics are also discussed, as this part is oriented toward people who know Oracle and are interested in migrating their applications to SQL Server 2000. The basics will also be beneficial for novice users of SQL Server 2000.

Part IV: Scripts and Utilities: Chapter 27 contains scripts and utilities to make the day-to-day task of database administration easy. All the scripts are explained and are included on the CD-ROM as well.

Part V: SQL Server.NET: .NET is the latest buzzword in the Microsoft arena. In Chapter 28 we take a look at how SQL Server 2000 fits into the .NET scenario and what can be expected in the next release of SQL Server, code named "Yukon."


About the Author

Rahul Sharma is working as a senior database administrator for Manhattan Associates Inc. in Atlanta, GA, and has been working with Microsoft SQL Server since SQL Server 6.5. He is currently working with SQL Server 2000 and Oracle 9i. He has more than five years of experience as a database administrator and has worked in Sybase 10, Oracle 7.1, 8.1.6, and 9i and SQL Server 6.5/7.0/2000. He also writes technical articles related to SQL Server for these two sites:

  • http://www.swynk.com (http://www.swynk.com/friends/sharma)

  • http://www.sqlservercentral.com (http://www.sqlservercentral.com/columnists/rsharma)


Acknowledgments

I want to thank many people for their support, without which I would not have been able to complete this book.

First and foremost, I would like to thank my father, Dr. C. L. Sharma, and my mother, Mrs. P. L. Sharma, for their encouragement and guidance. Whatever I have gained in life is due to the sacrifices they made.

I would like to thank my elder brother, Chetan Sharma, and my sister-in-law, Sarla Sharma, for their help and encouragement throughout my career and for believing in me.

A big "thank you" goes to my beautiful life partner, my wife Deepti. She was very understanding and helpful throughout this project. There were times when I thought this book would never get done, but she stood by me.

I also want to thank my wife's parents, Dr. Ved Shrotriya and Mrs. Dinita Shrotriya, and her brother, Harsh Vardhan Shrotriya, for their encouragement throughout the project.

Thanks are due to my colleague Venky Sivaraman, who has helped me in understanding the nuances of Oracle RDBMS. He is the best Oracle DBA I have ever worked with.

Special thanks to Ms. Alicia Carey, Ms. Mary T. O'Brien, and Mr. Michael Slaughter of Addison-Wesley for giving me the opportunity to work on this book. They were very supportive and patient throughout this project.


Chapter 1. SQL Server 2000 Overview

Microsoft SQL Server 2000 extends the performance, reliability, quality, and ease of use of Microsoft SQL Server 7.0. It includes several new features that make it an excellent database platform for large-scale, online transactional processing (OLTP), data warehousing, and e-commerce applications. The OLAP (online analytical processing) Services feature available in SQL Server 7.0 is now called SQL Server 2000 Analysis Services, and the term "OLAP services" has been replaced with the term "analysis services." Analysis services also includes a new data mining component. The Repository component available in SQL Server version 7.0 is now called Microsoft SQL Server 2000 meta data services and the term "meta data services" replaces the term "repository." The term "repository" is used only in reference to the repository engine within meta data services.


SQL Server Editions

There has been a lot of confusion over the name changes of the editions and the feature sets supported by the different editions of SQL Server 2000. Here is a list of what the editions were called before and what they are called now:

SQL Server 7.0 | SQL Server 2000
Desktop Edition | Personal Edition
Standard Edition | Standard Edition
Enterprise Edition | Enterprise Edition
MSDE | Desktop Engine
 | Developer Edition
 | CE Edition

For production servers, either the Standard or the Enterprise Edition is the logical choice since the Personal Edition is for PCs and small workgroups and the Developer Edition, which is the same as the Enterprise Edition, is licensed for use only as a development platform; the CE Edition is used as the data store on Windows CE devices. The main advantage(s) of using the Enterprise edition would be the increased support for physical memory, clustering of servers, more CPU SMP support, and distributed partitioned views. Table 1.1 compares the feature sets of the different editions of SQL Server 2000.

Note

The client software from all SQL Server editions, except for SQL Server CE, runs on any version of the operating system. Only the server components like the database engine and the analysis server are limited to the specific versions of the operating system. All the software from SQL Server CE runs exclusively on the Windows CE operating system. SQL Server 2000 also supports the following clients: Windows 95, Apple Macintosh, OS/2, and UNIX. Macintosh, UNIX, and OS/2 do not support the SQL Server GUI tools and require an ODBC driver from a third-party vendor like Merant/Intersolve.

Table 1.1. Operating systems supported by the different editions of SQL Server
Operating system | Enterprise Edition | Standard Edition | Personal Edition | Developer Edition | Desktop Engine | SQL Server CE
Microsoft Windows 2000 DataCenter | Supported | Supported | Supported | Supported | Supported | N/A
Windows 2000 Advanced Server | Supported | Supported | Supported | Supported | Supported | N/A
Windows 2000 Server | Supported | Supported | Supported | Supported | Supported | N/A
Windows 2000 Professional | N/A | N/A | Supported | Supported | Supported | N/A
Microsoft Windows NT 4.0 Server, Enterprise Edition | Supported | Supported | Supported | Supported | Supported | N/A
Windows NT 4.0 Server | Supported | Supported | Supported | Supported | Supported | N/A
Windows NT 4.0 Workstation | N/A | N/A | Supported | Supported | Supported | N/A
Microsoft Windows 98 | N/A | N/A | Supported | N/A | Supported | N/A
Microsoft Windows CE | N/A | N/A | N/A | N/A | N/A | Supported

Table 1.2 shows the number of processors that the database engine in different SQL Server 2000 editions can support on symmetric multiprocessing (SMP) computers.

Table 1.3 shows the maximum amount of physical memory, or RAM, that the database engine in Enterprise and Standard SQL Server 2000 editions can support.

Table 1.2. Maximum number of processors supported by the different editions
Operating system | Enterprise Edition | Standard Edition | Personal Edition | Developer Edition | Desktop Engine | SQL Server CE
Microsoft Windows 2000 DataCenter | 32 | 4 | 2 | 32 | 2 | N/A
Windows 2000 Advanced Server | 8 | 4 | 2 | 8 | 2 | N/A
Windows 2000 Server | 4 | 4 | 2 | 4 | 2 | N/A
Windows 2000 Professional | N/A | N/A | 2 | 2 | 2 | N/A
Microsoft Windows NT 4.0 Server, Enterprise Edition | 8 | 8 | 2 | 8 | 2 | N/A
Windows NT4.0 Server | 4 | 4 | 2 | 4 | 2 | N/A
Windows NT 4.0 Workstation | N/A | N/A | 2 | 2 | 2 | N/A
Microsoft Windows 98 | N/A | N/A | 1 | Use Desktop Engine | 1 | N/A
Microsoft Windows CE | N/A | N/A | N/A | N/A | N/A | 1

Table 1.3. Maximum amount of memory supported by the different editions
Operating system | Enterprise Edition | Standard Edition | Personal Edition | Developer Edition | Desktop Engine | SQL Server CE
Microsoft Windows 2000 DataCenter | 64 GB | 2 GB | 2 GB | 64 GB | 2 GB | N/A
Windows 2000 Advanced Server | 8 GB | 2 GB | 2 GB | 8 GB | 2 GB | N/A
Windows 2000 Server | 4 GB | 2 GB | 2 GB | 4 GB | 2 GB | N/A
Windows 2000 Professional | N/A | N/A | 2 GB | 2 GB | 2 GB | N/A
Microsoft Windows NT 4.0 Server, Enterprise Edition | 3 GB | 2 GB | 2 GB | 3 GB | 2 GB | N/A
Windows NT 4.0 Server | 2 GB | 2 GB | 2 GB | 2 GB | 2 GB | N/A
Windows NT 4.0 Workstation | N/A | N/A | 2 GB | 2 GB | 2 GB | N/A


Hardware Requirements for Installation

The minimum hardware requirements for running Microsoft SQL Server 2000 are listed in Table 1.4.

Table 1.4. Hardware requirements
Hardware | Minimum requirements
Computer

Intel or compatible

Pentium 166 MHz or higher.
Memory (RAM)[1]

Enterprise Edition: 64 MB minimum, 128 MB or more recommended Standard Edition: 64 MB minimum

Personal Edition: 64 MB minimum on Windows 2000, 32 MB minimum on all other operating systems

Developer Edition: 64 MB minimum

Desktop Engine: 64 MB minimum on Windows 2000, 32 MB minimum on all other operating systems
Hard disk space[2]

SQL Server database components: 95 to 270 MB, 250 MB typical

Analysis Services: 50 MB minimum, 130 MB typical

English Query: 80 MB

Desktop Engine only: 44 MB
Monitor

VGA or higher resolution

800x600 or higher resolution required for the SQL Server graphical tools
Pointing device | Microsoft Mouse or compatible
CD-ROM drive | Required

[1] Additional memory may be required, depending on operating system requirements.

[2] Actual requirements will vary based on your system configuration and the applications and features you choose to install.

Table 1.5. Comparison of the feature sets available in Standard and Enterprise editions of SQL Server
Feature | Benefit and description | Enterprise Edition | Standard Edition
E-Commerce and Internet features
Internet standards support | Integrate existing systems without programming through comprehensive support for W3C standards including XML, Xpath, XSL, and HTTP.
XML Views | View and access relational data using XML techniques by easily mapping XML elements and attributes to relational schema
URL and HTTP access | Flexibly access data from the Web via a URL. Use Structured Query Language (SQL), XML templates or Xpath in URLs for querying.
SELECT … FOR XML | Return XML from SQL queries and easily control the shape of the returned XML tree with three formatting options.
OpenXML | Access, manipulate, and update XML documents as if they were tables using T-SQL and stored procedures.
XML updategrams[1] | Use XML to selectively insert, update, and delete table data from anywhere, even through firewalls.
XML bulk load[1] | Seamlessly transmit, transform, and load data from any source into SQL Server 2000 relational tables.
Full-text search | Enable full-text search across the Web and intranets for formatted documents (e.g. Word, Excel, and HTML). Track changes automatically.
English Query | Allow all users to access data via natural language queries. Graphically author queries with wizards in the included Visual Studio environment. Generate MDX to query cubes.
Integrated data mining | Analyze collected relational and OLAP data including clickstreams and purchase histories to recognize trends and make predictions. Close the loop by delivering personalized content.
Linked cubes | Link cubes on different servers to supercharge analysis. Securely share or sell source cube data over the Internet.
HTTP access to cubes | Make cubes available through firewalls for Web-based analysis applications using an HTTP listener built into SQL Server 2000 Enterprise Edition.
Multi-instance support | Run reliably in hosted scenarios with separate database instances per customer or application.
Security | Protect data with higher default security on install. Includes support for secure sockets layer (SSL) connections and Kerberos. C2-level certification underway.
Installation disk imaging | Create standard or default databases for server farms easily and rapidly from any machine on the network with built-in cloning technology.
Scalability and reliability features
Distributed partitioned views | Achieve software scale out on the data tier by partitioning workload across servers. Add additional servers for greater scalability.
Log shipping | Automatically keep databases in synch for warm standby on multiple backup servers to share load—no matter how physically far apart.
Parallel index creation | Take full advantage SMP hardware to speed up index creation, easing the load on frequently updated systems.
Parallel scan | Accelerate scans on multiprocessor systems to improve overall database performance
Parallel DBCC | Verify database data quickly and efficiently with multiprocessor support.
Failover clustering | Install failover-ready databases directly from Setup. Use active-passive failover with standby hardware or active-active failover in hardware-constrained environments. Databases cam failover to any surviving node with 4-node failover.
Failover cluster management | Reinstall or rebuild any node in a failover cluster without impacting other nodes. Easily configure failover for replication and distributed partitioned views.
32 CPU SMP system support[2] | Scale-up SQL Server 2000 databases to SMP systems with as many as 32 processors.
64 GB RAM support[3] | Handle the largest data sets and transactional loads with up to 64 GB of RAM for SQL Server 2000.
VI SAN support | Speed communications between servers with SQL Server2000 support for direct communication with system area network (SAN) devices.
Indexed views | Create indexes on views to improve performance of existing queries without recoding. Speed up analysis and reporting that rely on complex views.
ROLAP dimension storage | Perform analyses on huge dimensions (10M+ members) by storing such dimensions relationally.
Distributed partitioned cubes | Easily use the Partition Wizard to separate one logical cube into separate physical partitions with complete transparency for flexible data storage and improved query performance.
Online index reorganization | Keep the server up and running while reorganizing indexes to improve performance.
Differential backup | Run backups quickly and with low server impact by backing up only changed pages.
Serverless snapshot backup[4] | Back up data seamlessly with effectively no impact on the database server. Restore quickly or create warm standby servers.
Management and development features
Active directory integration[5] | Manage databases centrally alongside other enterprise resources. View and search for servers, replication publications, cubes, and more.
Self-management and tuning | Rest easy as SQL Server 2000 uses dynamic algorithms to automatically adapt to constantly changing runtime environments.
Copy Database Wizard | Move and copy databases and objects between servers easily. Schedule migrations through integration with DTS.
SQL query analyzer | Debug stored procedures. Set breakpoints, define watches, view variables, and step through code. Trace executing code on server or client. Easily write T-SQL based on templates.
Data transformation services (DTSs) | Import and export primary and foreign keys between supported database products. Program the multiphase data pump and save DTS packages as Visual Basic code.
User-defined functions | Achieve code reuse by creating T-SQL functions. Incorporate routinely used logic to simplify development
Cascading referential integrity constraints | Control how changes propagate through tables when keys are updated.
INSTEAD OF and AFTER triggers | Execute code flexibly by specifying what happens in lieu of an operation or after it.
Indexes on computed columns | Define indexes on column types even when the data in the column is computed from other columns.
New data types | Store and reference data flexibly with bigint, sql_variant and table data types.
Column level collations | Store objects that have different collations in the same database. Collations can be specified at the database level or at the column level.
Integrated analysis services | Build OLAP, data warehousing, and data mining solutions to derive value from data.
Virtual cube editor | Browse, examine, and edit virtual cubes graphically.
MDX builder | Use drag-and-drop to quickly generate multidimensional expressions. Easily add functions, arithmetic operators, and more.
Dimensions | Pick the most appropriate dimension type with new parent-child, ragged, changing, write-enabled, and dependent dimensions.
Security in analysis services | Lock down sensitive data by defining security for dimensions and cells. Restrict roles to specific combinations of cells.
Custom rollups | Make sophisticated calculations directly from rollups with custom operators to specify how members affect parent values.
Multilanguage capabilities | Switch tools and user interfaces between supported languages on the fly.
OLAP actions | Allow analysis results to kick off processes in Web and business applications automatically.

[1] XML updategrams and XML bulk load are available from the Microsoft SQL Server home page as a Web release.

[2] Requires Windows 2000 DataCenter Server.

[3] Requires Windows 2000 DataCenter Server.

[4] Requires third-party virtual device interface (VDI) application like Veritas's NetBackup or BackupExec and enterprise storage system.

[5] Active Directory is a feature of the Windows 2000 server family.

The main advantage(s) of using the Enterprise edition would be the increased support for physical memory, clustering of servers, more CPU SMP support, and distributed partitioned views.


Installation of SQL Server 2000

SQL Server installations are generally very easy and don't present many problems. You need the correct operating system, the hardware requirements should match those prescribed by Microsoft, and IE5.0 must be installed because it is required for Microsoft Management Console (MMC) and HTML help. A minimal install of IE is sufficient, and IE is not required to be the default browser. Make sure that all other programs are stopped before you install SQL Server, since some of the programs might conflict with some files required by SQL Server. Also make sure that you stop all services that use ODBC since they can interfere with the MDAC (Microsoft Data Access Components) installation. I assume you have already worked with a previous version of SQL Server and are aware of the screens involved when you start installing SQL Server 2000, so instead of showing all the screen layouts and turning this book into a manual, I mention the steps where your install could go wrong and what to look out to ensure a successful install.

Essentially, the setup process consists of the following stages:

  1. Gather system data and requirements.

  2. Install MDAC.

  3. Install SQL Server.

  4. Run configuration scripts.

  5. Register the ActiveX components.

When SQL Server installation is complete, you will notice that the default security model it selected was Windows-only authentication. You can change that either through the Enterprise manager (right click on the server name, go to the security tab, and check SQL Server and Windows under Authentication) or in the registry (HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSSQLServer\MSSQLServer). Set the LoginMode to a value of 2.

Following are the steps for a successful install.

  1. On an NT box, make sure you have administrator-level permissions on the machine in question.

  2. Make sure that the server and workstation services are running. If the server service is not running, you may see "Error 2114 occurred while attempting to perform operation 'NetServergetInfo'." Look in the NT event log to ascertain the chain of events that has caused the services not to start.

  3. Make sure the machine is of the required spec to run the version of SQL you are installing (i.e., if you are installing SQL EE, make sure you have NT EE).

  4. If you have tried to install SQL before, then manually clean up all the files/registry entries as follows:

    1. Remove the <sql> directory and everything under it.

    2. Remove the <sql> directory from the path (use control panel/system).

    3. Remove the SQL registry entries using regedt32/regedit.

    If you have lost permission for any of the registry keys, then log on as administrator and take ownership of them starting at the highest relevant level. Use regedt32 and check the box to take ownership of all subkeys as well.

  5. SQL Server depends on network functionality even for the setup routines. Specifically, it needs named-pipe/mailslot functionality (on NT machines). Both usually require a network card to be present. If you don't have a network card, then go to Control Panel/Networks, choose Add Adapter, and then add the Microsoft loopback adapter—which is just a dummy driver, no hardware involved. It needs working network protocol(s) bound to it. Let whichever protocols you have use default parameters, except for tcp-ip. If you are using tcp-ip, do NOT specify dhcp assigned address; use 192.168.1.1 as the IP address and 255.255.255.0 as the subnet mask. This is a standard RFC1918 nonrouted IP address, so it shouldn't clash with any dial-up address you may be given by an ISP. If you are using the IPX/SPX protocol, accept the default frame type of 802.3. If the server service does not start after installing the loopback adapter and you get the message "The server service terminated with the following error: Not enough server storage is available to process this command" in the event log, then you need to reinstall your NT service pack to synchronize your NT networking files. One way of testing that the named-pipes functionality works is to run the makepipe and readpipe utilities that come with SQL Server. Run makepipe in one command window. Then run readpipe in another. If these work (that is, the readpipe connects to the makepipe and closes it down), then it is a good indication that networking functionality is ok.

  6. Because a named pipe, used by the setup routine, is effectively a file as far as the operating system is concerned, real-time virus scanners can cause problems. Most of these problems have been fixed so that they don't interfere with SQL's named pipes, so make sure you're running the latest version of whichever virus package you use. If you are in doubt, then disable the virus software for the duration of the install as it is a prime culprit for install errors/hangs.

  7. Other software packages/services can also interfere with SQL's install on NT systems. Typically they interfere with named-pipes/mailslot connectivity by intercepting requests. Setup will terminate with an error like "unable to write to mailslot … ." Shut down any/all of these packages for the duration of the install. Following are packages that are known to interfere with SQL's setup routines:

    1. IIS (Internet Information Server)—the Web parts, not FTP

    2. PWS (Personal Web Server)

    3. Exchange Server

    4. SNA Server

    5. Oracle

    6. DBWeb

    7. Backup software, e.g., Arcserve/Backup Exec

    8. Systems management, e.g., UniCentre, Compaq Insight Manager

    9. Microsoft SMTP

    10. Microsoft NNTP

    11. Disk Keeper

    12. Protected Storage (part of Internet Explorer)

    13. SNMP Service(s)

    On Win9x systems you may also get problems if the Quicken Download Manager is running.

  8. If the dial-up networking icon/window appears and tries to make a network connection, then stop and disable the Remote Access Autodial Manager via Control Panel/Services. This is an NT issue rather than an SQL issue, but disabling autodial is the easiest way around it.

  9. If you are upgrading, make sure that the default database for the sa login is master.

  10. If the error is to do with SQL Performance Monitor counters—SQLCTRxx.DLL—then try removing the SNMP service and all third-party network/server monitoring tools, for example, HP NetServer Agents, Compaq Insight Manager.

  11. If you are getting a gpf or registry error installing on NT, it could be the known issue with SNMP caused when SNMP managers that come with server management tools (especially HP servers) register a lot of SNMP extensions, specifically when HKLM\System\CCS\Services\SNMP\Paramaters\ExtensionAgents get to around 1K or more. Either remove the SNMP service before installing SQL Server, or try the following workaround:

    1. Stop SNMP service.

    2. Save the contents (all the values) of the mentioned subkey (use regedt32 for these operations).

    3. Delete all values except the one with the "largest" name (the names are composed of digits).

    4. Install SQL Server.

    5. Check the extensionagents subkey. Setup will have added a value entry for SQL Server. Save the value name and value somewhere so that you can manually add it again later.

    6. Restore all extensionagents values you just saved.

    7. Add the sql server entry in case it got nuked in the previous step.

    8. Restart snmp.

  12. If the problem is to do with ODBC files not installing due to being "in use"—a typical SQL 7 problem—then check that nothing has the ODBC files open. This could be other SQL utilities, Exchange, IIS, SNMP Service, Compaq Insight Manager, Backup Exec, Microsoft Netshow Program Server, Terminal Server License Server, and many other services. Download NTHANDLE/X from www.sysinternals.com (free) if you can't figure out what has the file(s) open; check View dll's, then go into the search function and look for odbc [image].dll. It could also be that the ODBC files on your hard drive have been marked as read-only. Check with ATTRIB.EXE or Explorer to see if this is the case.

  13. If the message is "Critical error, could not open the file named D:\MSSQL\BINN\SQLCTRxx.DLL," then first just retry, as the performance monitor could have just a temporary lock on the file.

  14. If the .OUT files indicate that ISQL cannot connect to the SQL Server to run scripts, then you can try the following (it isn't supported, and I have used it only to fix service pack installs so far, but it may work for full installs as well). After making the change run "setup /t Local = Yes," make a backup of the setup.inf first, then find the following line: set !ServerName = $(!ComputerName). Now add the following after it: ifstr(i) $(!Local) == "Yes" set !ServerName = "." endif

  15. For upgrades, make sure that SQL Server has a name. Run sp_helpserver and check what name srvid of 0 has. If it is not there then run sp_addserver '<servername>', 'LOCAL'

  16. Getting a 109 error when you start SQL Executive is a permissions problem with the NT account that you supplied, which does not have permissions to the relevant service or registry keys—HKLM\SOFTWARE\MICROSOFT\MSSQLSERVER. The easiest thing to do is install both MSSQLSERVER and SQLEXECUTIVE with the local system account and then go back and change it later.

  17. If you get a 432 error, check the file "uninst.exe" in the <WINNT> directory. Make sure that you have permission to do so, and take ownership of it if necessary.

  18. Once installed, if you have trouble registering your SQL Server by name in SQL EM, then just register it with a name of just a period. This again bypasses the network layer and ensures that a local named pipe is used.

Debugging Installation Issues with SQL Server 2000

If the setup fails during the process, the Install shield will partly roll back the installation. You will need to delete the SQL Server directory and the registry key (HKEY_LOCAL_MACHINE\SOFTWARE\MICROSOFT\Microsoft SQL Server). Do not reinstall until you have removed the registry key and the directory and rebooted the server.

When the install shield fails, first look at the SQL Server error logs to see where it is actually failing. These can be found in three locations. The first place to look is in the main SQL Server logs directory (<drive Letter>:\Program Files\Microsoft SQL Server\MSSQL\LOG by default). This log will tell you if SQL Server is failing on startup. It will have information on why the setup failed. The second place to look at is the C:\WINNT\Sqlstp.log. Look for "errors." The third place to look is the NT Application event log.

The setup is most likely to fail during the configuration step when SQL Server places the database server in a single-user mode and runs scripts to configure the system catalog. Make sure that no other connections are hitting your server during this process. Check whether the files were copied over and also whether the service was created. If both of these are correct, go to the DOS prompt (go to the Binn directory for SQL Server) and run the following command:

OSQL /Usa /P<password> /S<server_name>

Once you see a 1>, then run any sql command, such as

1>Select @@version
2>GO

If you see results from this, you can determine that your service is functioning and you are failing during the configuration step.

If there is a problem with the registry, you can rebuild the registry during the installation process. This option is new in SQL Server 2000 and can be accessed by rerunning the setup.

  1. Run SQL Server setup and pick the "Advanced Installation" option.

  2. Select "Registry Rebuild."

  3. Go through all setup options.

Uninstall SQL Server and reboot and install again.

Sometimes when you uninstall and reinstall SQL Server from the control panel (Add/Remove Programs), the entry from the registry is not completely removed. You need to remove the entry from HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SessionManager. Under this key, you will see an entry called PendingFilesRenameOperations. You need to delete this and then install SQL Server. If you uninstall SQL Server using the CD, then this problem does not surface.

Upgrading from a Previous Version of SQL Server

We can upgrade from a previous version of SQL Server and can also do an upgrade after SQL Server 2000 is installed. Upgrades from SQL Server 6.5 and from SQL Server 7.0 are different. With SQL Server 2000, you can have multiple instances on the same box. SQL Server 6.5 databases are converted to the SQL Server 2000 formats and SQL Server 7.0 installation is overwritten by SQL Server 2000 unless a named instance of SQL Server 2000 is installed, in which case both versions can coexist but the client tools will be the ones that are installed from SQL Server 2000. The SQL Server 2000 SQL-DMO (the object model on which the Enterprise Manager is based; see Chapter 20) is backward compatible, so you can use it for administering SQL Server 7.0 installations as well.

Upgrading from SQL Server 6.5 to SQL Server 2000.

After installing SQL Server 2000, you can upgrade your SQL Server version 6.5 server configuration and databases using the SQL Server Upgrade Wizard. In this process, SQL Server 6.5 information and data is transferred to a SQL Server 2000 installation. The SQL Server 6.5 installation remains intact on the computer. It can be removed or kept on the computer and used in a version switch configuration with SQL Server 2000. In case you plan to keep SQL Server 6.5 installed along with an instance of SQL Server 2000, Service Pack 5 must be applied to SQL Server 6.5 before installing SQL Server 2000. The reason you might want to have different versions of SQL Server installed could be to maintain version-dependent code and to benchmark your system performance on the two versions.

Upgrading from SQL Server 7.0 to SQL Server 2000.

You can overwrite your existing installation of SQL Server 7.0 by installing a default instance of SQL Server 2000. You can also keep your installation of SQL Server 7.0 intact by installing a named instance of SQL Server 2000. Both operations are performed using the following steps.

  1. Insert the Microsoft SQL Server 2000 compact disc for the edition to which you want to upgrade into your CD-ROM drive. If the compact disc does not autorun, double-click Autorun.exe in the root directory of the compact disc. (Note: If you have purchased an edition of SQL Server that has more features than your current SQL Server 7.0, the upgrade process will perform both the version and edition upgrade at the same time.)

  2. Select SQL Server 2000 Components, select Install Database Server; setup then prepares the SQL Server Installation Wizard. At the Welcome screen, click Next.

  3. In the Computer Name dialog box, Local Computer is the default option and the local computer name appears in the Edit box. Click Next.

  4. In the Installation Selection dialog box, click "Upgrade, remove, or add components to an existing instance of SQL Server," and then click Next. In the Instance Name dialog box, Default will be selected. Click Next. (Note: When upgrading, SQL Server 7.0 automatically becomes the default instance of SQL Server 2000.)

  5. In the Existing Installation dialog box, click "Upgrade your existing installation," and then click Next.

  6. In the Upgrade dialog box, you are prompted as to whether you want to proceed with the requested upgrade. Click "Yes, upgrade my <text specific to the upgrade>" to start the upgrade process, and then click Next. The upgrade runs until finished.

  7. In the Connect to Server dialog box, select an authentication mode and then click Next. If you are not sure which mode to use, accept the default: The Windows account information I use to log on to my computer with (Windows).

  8. In the Start Copying Files dialog box, click Next.

  9. In the Setup Complete dialog box, click "Yes, I want to restart my computer now." Then click finish.

Edition Upgrades

Each of the editions of Microsoft SQL Server 2000 (Enterprise, Standard, Personal, and Developer) comes with its own compact disc and is installed separately. You can add to the feature set of your installation of Microsoft SQL Server by performing a version and edition upgrade at the same time (Table 1.6). For example, if you have purchased Microsoft SQL Server 2000 Standard Edition, you can upgrade from SQL Server 7.0 Desktop Edition to SQL Server 2000 Standard Edition.

You can add functionality while upgrading from one edition of SQL Server 2000 to another, and you have the option to add components during this upgrade. For example, if you upgrade from the SQL Server 2000 Personal Edition to the SQL Server 2000 Standard Edition, you can install additional components during the upgrade process.

To upgrade a SQL Server 2000 installation to a different edition of SQL Server 2000, take the following steps.

Table 1.6. Upgrade options between versions and editions of SQL Server
Installed version | Can upgrade to
SQL Server 7.0 Desktop Edition

SQL Server 2000 Personal Edition

SQL Server 2000 Standard Edition
SQL Server 7.0 Standard Edition

SQL Server 2000 Standard Edition

SQL Server 2000 Enterprise Edition
SQL Server 7.0 Enterprise Edition | SQL Server 2000 Enterprise Edition
SQL Server 2000 Developer Edition | SQL Server 2000 Enterprise Edition
SQL Server 2000 Standard Edition | SQL Server 2000 Enterprise Edition
SQL Server 2000 Personal Edition | SQL Server 2000 Standard Edition

  1. Insert the Microsoft SQL Server 2000 compact disc for the edition you want to install into your CD-ROM drive. If the compact disc does not autorun, double-click Autorun.exe in the root directory of the compact disc.

  2. Select SQL Server 2000 Components, select Install Database Server, and then setup prepares the SQL Server Installation Wizard. At the Welcome screen, click Next.

  3. In the Computer Name dialog box, select Local Computer or Remote Computer.

  4. In the Installation Selection dialog box, click Upgrade, Remove, or "Add components to an existing instance of SQL Server," and then click Next.

  5. In the Instance Name dialog box, click Next.

  6. In the Existing Installation dialog box, click "Upgrade your existing installation," and then click Next.

  7. If Setup detects that you are doing an edition upgrade, the Upgrade dialog box appears. Click "Yes, upgrade my <text specific to the upgrade>" to upgrade the feature set of your current installation, and click Next.

  8. After the upgrade is completed, you are prompted as to whether you want to install additional components. If you click Yes, the Select Components dialog box appears. Accept the defaults or select the additional components you want to install, and then click Next.

  9. You can select an item in the components list and then select items from the related subcomponents list. Click to select items you want to install; clear the checkboxes of the items you do not want to install.

  10. When you are finished specifying options, in the Start Copying Files dialog box, click Next.

In the Setup Complete dialog box, click "Yes, I want to restart my computer now," and then click Finish to complete the edition upgrade.

Adding Components to an Instance of SQL Server 2000

You can add components to an existing installation of SQL Server 2000 by selecting the instance of SQL Server to which you want to add features. This method allows you to have different component sets for the default instance and for any named instances. (Note: You cannot remove components by clearing checkboxes in the Select Components dialog box. If you need to remove components from an instance of SQL Server, you must uninstall the instance.)

To add components to an instance (default or named) of SQL Server 2000, follow these steps.

  1. Run SQL Server Setup, select SQL Server 2000 Components, select Install Database Server, and then click Next at the Welcome screen of the SQL Server Installation Wizard.

  2. In the Computer Name dialog box, Local Computer is the default option and the local computer name appears in the edit box. Click Next.

  3. In the Installation Selection dialog box, click "Upgrade, Remove, or Add components to an existing instance of SQL Server," and then click Next.

  4. In the Instance Name dialog box, Default is selected if you have the default instance installed.

  5. If you want to add components to a named instance, select it from the Instance Name list, and then click Next.

  6. In the Existing Installation dialog box, click "Add components to your existing installation," and then click Next.

  7. In the Select Components dialog box, select a component from the components list, and then select items from the related subcomponents list. Click to select items you want to add, and then click Next.

  8. For information about each component, select the item and view the Description box.

  9. When you are finished specifying options, click Next in the Start Copying Files dialog box to add components to the selected instance of SQL Server.

Uninstalling SQL Server 2000

To uninstall an existing installation of SQL Server 2000 (default or named instance), take the following steps.

  1. Insert the Microsoft SQL Server 2000 compact disc into your CD-ROM drive. If the compact disc does not autorun, double-click Autorun.exe in the root directory of the compact disc.

  2. Select SQL Server 2000 Components, select Install Database Server, and then setup prepares the SQL Server Installation Wizard. At the Welcome screen, click Next.

  3. In Computer Name dialog box, select Local Computer or Remote Computer.

  4. In the Installation Selection dialog box, click "Upgrade, Remove, or Add components to an existing instance of SQL Server," and then click Next.

  5. In the Instance Name dialog box, Default is selected if you have the Default instance installed. If you want to uninstall a named instance, select it from the Instance Name list box, and then click Next.

  6. In the Existing Installation dialog box, click "Uninstall your existing installation," and then click Next.

  7. Setup removes the selected installation. In the Uninstall dialog box, click Next, and then in the Setup Complete dialog box, click Finish.

Unattended Installs

You can perform an unattended installation of Microsoft SQL Server 2000, in which setup screen entries are made automatically using stored information. An unattended installation can be convenient if you want to perform several installations of SQL Server with identical configurations on different computers. An unattended installation requires a setup initialization file, which can be created in several different ways.

By default, each time you install SQL Server using the Setup screens, the options you select are recorded into the setup initialization file, Setup.iss. Setup.iss is placed in the system root directory (%windir%) and is available for installation settings at a later time. (Note: You cannot perform an unattended installation to set up a failover cluster of Microsoft SQL Server 2000.)

In Setup, when you select the "Record unattended .iss" file option in the Advanced Options screen, each subsequent choice you make in the setup screens is recorded in the Setup.iss file stored in the system root directory. SQL Server files are not installed in this process. The Setup.iss file can then be run as is, or revised in a text editor if necessary.

The Record unattended setup option allows you to simulate an installation and create an .iss file that can be used later for an unattended installation of Microsoft® SQL Server™ 2000. SQL Server files are not installed in this process.

To create a file for an unattended installation, follow these steps.

  1. Insert the Microsoft SQL Server 2000 compact disc into your CD-ROM drive. If the compact disc does not autorun, double-click Autorun.exe in the root directory of the compact disc.

  2. Select SQL Server 2000 Components, select Install Database Server, and then click Next at the Welcome screen of the SQL Server Installation Wizard.

  3. In the Computer Name dialog box, select the option you want, and click Next.

  4. In the Installation Selection dialog box, click Advanced Options, and then in the Advanced Options dialog box, click "Record unattended .iss file." Click Next.

  5. In subsequent Setup screens, select the options you want for the unattended installation. After you finish selecting the options, click Next in the Setup Information screen.

  6. In the Setup Complete screen, click Finish.

This message appears: "Setup has collected the information needed to create an unattended installation file (.iss) for use with later unattended installations of SQL Server." SQL Server Setup then creates the Setup.iss file in the %windir% location with the options you have specified.

You can run an unattended installation by using sample batch files and setup initialization files included on the Microsoft SQL Server 2000 compact disc. Or you can run the Setup program directly from the command prompt in the appropriate directory for the edition of SQL Server you want to install, using arguments as needed.

To run an unattended installation using ready-made batch files:

  1. Locate the .bat and .iss files in the root directory of your SQL Server 2000 compact disc.

  2. View the .bat and associated .iss files, and modify if necessary.

  3. Run the appropriate batch and setup files from the command prompt:

  4. For a standard unattended installation, run Sqlins.bat.

  5. For a client-only unattended installation, run Sqlcli.bat.

  6. For a custom unattended installation, run Sqlcst.bat.

  7. To run an unattended installation directly from the command prompt, run Setupsql.exe from the Setup directory in the appropriate architecture directory.

Use arguments as needed:

-f1 <initialization file path>: Selects an unattended setup initialization file.

start/wait command (with the -SMS option): Returns control to the command prompt only after SQL Server Setup completes.

-s flag: Causes the setup program to run in silent mode with no user interface.

Installing SQL Server Using SMS

You can use Microsoft Systems Management Server (SMS) version 1.2 or later to install Microsoft SQL Server 2000 automatically on multiple server computers running Microsoft Windows NT or Microsoft Windows 2000. The SQL Server compact disc contains a package definition format (PDF) file (Smssql.pdf) that automates creating an SQL Server package in SMS. The SQL Server package can then be distributed and installed on SMS computers. Smssql.pdf includes instructions for running the batch file Smssqins.bat with Sqlins.iss (the setup initialization file) for a typical installation. Both of these files are included on the SQL Server compact disc. To create a custom command file, edit a copy of Smssql.pdf.

You create a customized setup initialization file interactively when you select the "Record unattended .iss file" option in Microsoft SQL Server 2000 Setup. You can also edit files manually, to further refine and customize setup initialization files.

You can use a text editor to modify the Setup.iss file generated using the "Record unattended .iss file" option. You can also modify one of the sample setup files ( [image].iss) included on the SQL Server 2000 compact disc or you can create your own setup file.

To modify one of the sample setup initialization files found on the SQL Server compact disc, open the file in a text editor and modify as required. Keep the file compatible with the Microsoft Windows initialization file format and save it with the .iss file name extension.

A setup initialization file is a text file that uses the standard Windows .ini file format. The sample setup initialization file (Sqlins.iss) is found in the root directory of the SQL Server compact disc. (Note: In creating a setup file for a named instance of SQL Server 2000, you must indicate the instance name you want to install and the path required to navigate through the setup screens. The Instance Name dialog box [DlgInstanceName] section must be modified; so must other places in the setup file where the instance name appears.)

Rebuilding the Registry

The Registry Rebuild option on the Advanced Options Setup screen allows you to rebuild the registry for a corrupted Microsoft SQL Server installation. This process fixes only the registry; it does not fix data errors or the master database.To rebuild the registry, follow these steps.

  1. Insert the Microsoft SQL Server 2000 compact disc into your CD-ROM drive. If the compact disc does not autorun, double-click Autorun.exe in the root directory of the compact disc.

  2. Select SQL Server 2000 Components, select "Install database server," and then click Next at the Welcome screen of the SQL Server Installation Wizard.

  3. In the Computer Name dialog box, click Next.

  4. In the Installation Selection dialog box, click Advanced Options, and then in the Advanced Options dialog box, click Registry Rebuild. Click Next.

  5. A message appears informing you that Setup rebuilds the registry based on information you supply in the subsequent screens.

Caution

The setup options you enter must be the same choices you entered during the initial installation. If you do not know or are not sure of this information, do not use the registry rebuild process. Instead, uninstall and reinstall SQL Server.

Note

Rebuilding the registry includes recopying external components such as MDAC and MS DTC.


Multiple Instances

Beginning with SQL Server 2000, you can have multiple instances of the SQL Server engine running on your machine. Each instance operates independently from any other instance on the same computer, and applications can connect to any of the instances. The number of instances that can run on a single computer depends on resources available. The maximum number of instances supported in SQL Server 2000 is 16. When installing a new version or edition of SQL Server 2000 or maintaining an existing installation, you can specify it as a default instance or a named instance of SQL Server. When you install SQL Server 2000 on a computer with no existing installations of SQL Server, Setup specifies the installation of a default instance. However, you can choose to install SQL Server 2000 as a named instance instead by clearing the Default option in the Instance Name dialog box.

A named instance of SQL Server is identified by the network name of the computer plus an instance name, in the format <computername>\<instancename>. Applications must use SQL Server 2000 client components to connect to a named instance. A computer can run any number of named instances of SQL Server concurrently. A named instance can run at the same time as an existing installation of SQL Server version 6.5 or SQL Server version 7.0. The instance name cannot exceed 16 characters.

A new instance name must begin with a letter, an ampersand (&), or an underscore (_) and can contain numbers, letters, or other characters. SQL Server sysnames and reserved names should not be used as instance names. For example, the term "default" should not be used as an instance name because it is a reserved name used by Setup.

Single and multiple instances of SQL Server 2000 (default or named) are available using the SQL Server 2000 Personal Edition, the SQL Server 2000 Standard Edition, or the SQL Server 2000 Enterprise Edition.

Default Instances

A default instance of SQL Server is identified by the network name of the computer on which it is running. Applications using client software from earlier versions of SQL Server can connect to a default instance. SQL Server version 6.5 or SQL Server version 7.0 can operate as default instances. However, a computer can have only one version functioning as the default instance at a time.

You cannot install a default instance of SQL Server 2000 on a computer that is also running SQL Server 7.0. You must either upgrade the SQL Server 7.0 installation to a default instance of SQL Server 2000, or keep the default instance of SQL Server 7.0 and install a named instance of SQL Server 2000. You can install a default instance of SQL Server 2000 on a computer running SQL Server 6.5, but the SQL Server 6.5 installation and the default instance of SQL Server 2000 cannot be running at the same time. You must switch between the two using the SQL Server 2000 vswitch command prompt utility.

Named Instances

A named instance of SQL Server 2000 can be installed at any time: before installing the default instance of SQL Server 2000, after installing the default instance of SQL Server 2000, or instead of installing the default instance of SQL Server 2000.

Each named instance is made up of a distinct set of services and can have completely different settings for collations and other options. The directory structure, registry structure, and service names all reflect the specific instance name you specify.

File Locations for Multiple Instances of SQL Server

Each named instance of Microsoft SQL Server 2000 has a specific location for its program files and another for its data files that is different from that of the default instance of SQL Server. For each named instance of SQL Server that you install, the default directories are

\Program Files\Microsoft SQL Server\MSSQL$InstanceName\Binn

for executable files and

\Program Files\Microsoft SQL Server\MSSQL$InstanceName\Data

for data files.

Shared tools for all instances, both default and named instances, are located in the directory

\Program Files\Microsoft SQL Server\80\Tools

You can specify file paths other than the default locations for program and data file for multiple instances. (Note: If Microsoft SQL Server version 7.0 is used as the default installation alongside a named instance of SQL Server 2000, program and data files are located at C:\Mssql7, the default location for SQL Server 7.0 files.)

If you are uncertain about instance paths, query the registry to get the installation path of a particular instance. Run the following at the command prompt, inserting the appropriate instance name:

C:\> REG QUERY HKLM\Software\Microsoft\Microsoft SQL
Server\InstanceName\MSSQLServer\Setup\SQLPath

You can also use the USP_Registry_Info script in Chapter 27.


Network Protocols

When you install a default instance of Microsoft SQL Server 2000, the standard network addresses are enabled. For example, named pipes uses \\.\pipe\sql\query, and TCP/IP sockets connect to port 1433.

When you select a named instance, only the named pipes, TCP/IP, and NWLink IPX/SPX protocols are supported. Named pipes defaults to a network address of \\Computername\Pipe\MSSQL$instancename\Sql\Query. The port addresses used by TCP/IP and NWLink IPX/SPX are chosen dynamically (by default) the first time the instance is started.


Collations Support for Internationalization

In Microsoft SQL Server 2000, it is not required to separately specify code page and sort order for character data and the collation used for Unicode data. Instead, specify the collation name and sorting rules to use. The term "collation" refers to a set of rules that determine how data is sorted and compared. Character data is sorted using rules that define the correct character sequence, with options for specifying case sensitivity, accent marks, kana character types, and character width.

Windows collations define rules for storing character data based on the rules defined for an associated Windows locale. The base Windows collation rules specify which alphabet or language is used when dictionary sorting is applied, as well as the code page used to store non-Unicode character data. SQL collations are provided for compatibility with sort orders in earlier versions of Microsoft SQL Server. Select this option to match settings compatible with SQL Server version 7.0, SQL Server version 6.5, or earlier.

When you set up SQL Server 2000, it is important to use the correct collation settings. You can change collation settings after running Setup, but you must rebuild the databases and reload the data. It is recommended that you develop a standard within your organization for these options. Many server-to-server activities can fail if the collation settings are not consistent across servers.

Use the Collation Settings screen to modify default collation settings. Use the Windows Locale option to match collation settings in instances of Microsoft SQL Server 2000. Use SQL Collations to match settings that are compatible with the sort orders in earlier versions of SQL Server.

Change the default settings for Windows Locale (Windows collation) only if your installation of SQL Server must match the collation settings used by another instance of SQL Server 2000 or the Windows locale of another computer.

Select the name of a specific Windows collation from the list. For example, use Latin1_General for the U.S. English character set (code page 1252); use Modern_Spanish for all variations of Spanish, which use the same character set as U.S. English (code page 1252); use Arabic for all variations of Arabic, which use the Arabic character set (code page 1256).

Select Sort Order options to use with the collation designator selected. Binary is the fastest sorting order, and is case-sensitive. If Binary is selected, the Case-sensitive, Accent-sensitive, Kana-sensitive, and Width-sensitive options are not available.


Checklist for a Successful Install of SQL Server

This checklist can be used for making sure that the SQL Server install was sucessful and the parameter settings for the server and the databases are what they should be. The checklist can also be used as an audit sheet for the servers over a period of time to make sure that they are configured correctly and everything is working as planned.

Server Information
Machine Name: | Service Pack(s)
Processor: | Location:
Memory: | Setup Date:
IP Address: | Operating System

Controller/Disk Configurations
Disks | Capacity | Controller | Channels

RAID Configurations
Array no. | Type | Disks | Logical letter | Channel | Capacity (MB)

Checklist
Item | Status | Action required
Sql server edition |  |
SQL Server Service Pack |  |
Memory allocation |  |
Instances |  |
Databases per instance |  |
Filegroups and distribution of the files |  |
Datafiles (size and placement) |  |
Log files (size and placement) |  |
Temp DB (size and placement) |  |
System databases placement |  |
Server options configuration |  |
Database options configuration |  |
Recovery model |  |
Backups (type, frequency, and disk backups/tape backups) |  |
Maintenance and integrity checks |  |
Scheduled scripts |  |
Clustering |  |
Log shipping configuration |  |
Third-party software |  |
Purges (frequency) |  |
Network protocols |  |
Sql server licenses |  |
Number of concurrent users |  |

You may notice that some of the options, like the server settings and the database settings options, are pretty broad. You might have selected these options at the time of setting up the servers and the database; some of these options might no longer be necessary; or you may want to review the options as your database load increases. Some of the database options follow.

AutoClose | : False
Select Into/Bulk Copy | : False
Trunc. Logon Chkpt. | : True
Read Only | : False
DBO Use Only | : False
Single User | : False
AutoShrink | : False
ANSI NULL Default | : False
Recursive Triggers | : False
Ansi NULLS | : False
Concat Null Yields Null | : False
Cursor Close on Commit | : False
Default to Local Cursor | : False
Quoted Identifier | : True
ANSI Warnings | : False
Auto Create Statistics | : True
Auto Update Statistics | : False
Offline | : False
Torn Page Detection | : True


Maintenance and Integrity Checks

Here are some of the maintenance routines that you should schedule in order to ensure the integrity and safety of your data and to ensure good performance:

Defrag the database.

Over a period of time, because of the DML statements (updates, deletes, and inserts), the data and the index pages can become fragmented. You can use the USP_DefragDatabase sql script shown in Chapter 27 to defrag the database. It is recommended that it be run every week for production database(s).

Shrink the production database

The autoshrink option for the production database(s) should be turned off; if it's on, then the SQL Server query engine will go and check every half an hour or so whether it needs to shrink the database. The DBCC SHRINKDATABASE (database_name) command should be scheduled to be run every night at off-peak hours.

Update statistics on the production database

There are really two options over here. One is to switch the autostats option on. The drawback in this case is that the SQL Server query optimizer checks after every 600–700 DML statements whether it needs to update the statistics that can have bad impact on the OLTP database(s) depending on what your traffic is like. The other option is to switch the autostats option off and schedule one of the scripts in the Update_Stats SQL shown in Chapter 27.

Run DBCC checks on the database

DBCC (database consistency checks) statements should be run against the database(s) on the production or test environment in order to check for the inconsistencies that may exist between the user tables or the system tables. DBCC CHECKDB and DBCC CHECKCATALOG are the DBCC checks that should be run.

Back up the production and test database(s)

It is a good policy to perform hot backups regularly. If the database crashes, regular backups will allow recovery of all transactions up to the point of the time of the crash. However, this option does require regular monitoring and maintenance. If a hot backup is not possible, a cold backup needs to be performed. You can build up a backup plan in order to secure your data according to your needs. I would recommend doing full database backups weekly, differential database backups (incremental backups) daily and transaction log backups 2 to 3 times a week. This of course depends upon the volume of data you have. You can use the VBScript (BackUp.vbs) that is supplied as a part of the scripts in Chapter 27 for scheduling the backups.


Summary

In this chapter we covered the different editions of SQL Server, the differences between the Standard and the Enterprise editions, the installation steps, and where to look for errors if the install is not successful.

We also looked at the different collations that you can choose from, how you can do an upgrade to SQL Server 2000, and how to perform unattended installs. We took a look at the differences between the named instances and the multiple instances and the file placements in the case of multiple instances.


Part I: New Features and Improvements over SQL Server 7.0

In this part, we outline the feature sets that have been enhanced since SQL Server 7.0 and the features sets that have been added in SQL Server 2000 like XML capability, indexed views, distributed partitioned views, and so on. We will first outline these features and then delve more into them by giving examples on each of these feature sets in the subsequent chapters.


Chapter 2. Enhancements and Changes to Existing Features

SQL Server 2000 is touted as an integral piece of the Internet platform (see the detailed discussion on .NET in Chapter 28 ) to build and deploy solutions that integrate customers, partners, business processes, and applications. .NET is the comprehensive, integrated platform for building and deploying applications for the Business Internet. .NET includes Web application services, RAD development tools, and scalable server applications for creating Web-based solutions, with Windows 2000 as the foundation.


Feature Sets

The descriptions here should give you some idea about these feature sets of SQL 2000. They include some of the key features that are useful for application development as well as features for administering the servers. Part I goes into detail about the enhancements to the feature sets of SQL Server 7.0, and Part II discusses the details of the new features added in SQL Server 2000.

Fully Web-Enabled

SQL Server 2000 allows the use of XML to deploy systems tha t loosely interchange data, and it supports W3C standards. XML support can be provided in SQL Server 7.0 by writing extended stored procedures and standard stored procedures, although standard stored procedures can drag down performance for large data sets with a complex structure. In addition, certain SQL 7.0 functions, such as full-text search, permit XML to be stored as text. What features make SQL Server 2000 officially XML-enabled? Traditionally, you can request two kinds of XML from a database: static XML stored within the database and dynamic XML generated from data within the database. Even SQL 7.0 could serve as an XML repository, but SQL Server 2000 has a rowset provider that is capable of mapping over an XML DOM (within a stored procedure). The data must be passed as text and is mapped into normal SQL Server 2000 data-types. SQL Server 2000 also permits use of the following.

  • URLs to access SQL server through HTTP

  • FOR XML clause in the SELECT statement to retrieve results in XML format

  • System stored procedures for manipulating XML data

  • Updategrams for batch operations

  • SQL Server OLE DB provider extensions for XML

English Query

SQL 2000 helps in simplifying application development and deployment by using English Query in the Web application. English Query is a feature that enables creation of applications that accept queries in everyday English rather than in complex SQL statements. English Query consists of an authoring tool and an engine. To implement natural language searching, we first use the authoring tool to provide domain knowledge to the engine. The authoring tool makes it easy to relate database entities to objects in the domain. For example, we might create a verb relationship between salespeople and products by indicating "Salespeople sell products." English Query uses these relationships to perform true, deep, natural language parsing of users' questions, which provides better search results than does keyword-based technology. The engine converts English questions into standard Structured Query Language (SQL) statements and then sends these statements to the database. The engine also processes responses from the database.

English Query can be used in any environment that supports COM. In particular, English Query comes with sample scripts that help to set up natural language searching on the Web. This feature makes an application more user friendly and facilitates ease of user queries.

More Scalable and Reliable

SQL Server 2000 delivers a broad range of scalability and reliability features.

Scalability for e-commerce solutions

With SQL 2000 an application can take full advantage of SMP (symmetric multiprocessing) hardware. It supports four servers in a clustered environment compared to two with SQL Server 7. Additionally, SQL 2000 takes full advantage of Windows 2000, including support for the Active Directory service and up to 32 processors and 64 GB of RAM (Enterprise Edition); the limits will be even higher when the 64-bit version ships late in 2001. The Standard Edition can be used on symmetric multiprocessing systems with up to 4 CPUs and 2 GB of RAM.

In the longer term, there are additional significant technology milestones ensuring an accelerating expansion of scalability for customers building and deploying solutions on the Microsoft platform. These innovations include the introduction of Intel Pentium III Xeon-based servers with 16 and 32 processors, offering many times more power than existing 8-processor systems. In addition, following the introduction of Intel's 64-bit Itanium platform later in 2000, Microsoft will release 64-bit versions of Windows 2000 and SQL Server 2000. These state-of-the-art systems will allow customers to utilize terabytes of system memory for the most complex applications.

Maximized uptime and reliability

SQL 2000 simplifies failover cluster configuration and management and allows a database to remain online during most operations. It enables differential and "serverless" snapshot backups.

BizTalk Server 2000

A new XML-based business-to-business commerce solution, BizTalk Server 2000 provides business process integration within the enterprise and with trading partners across the Internet through the exchange of XML-formatted business documents. SQL Server provides high-performance storage and easy-to-schedule transformation capabilities for data from BizTalk Server. SQL Server 2000 and BizTalk Server 2000 support the same XML data reduced schema. This allows documents to be transmitted directly from SQL Server to BizTalk Server and vice versa.

Multi-Instance Support

Multi-instance support permits multiple, isolated installations of SQL Server on a single machine. Earlier versions of SQL could host multiple databases on the same server, but in practice, only one server instance could be active at a time. Multi-instance support not only enables basic tasks, such as maintaining two independent SQL Server environments for development and testing teams on the same physical machine, it also makes it easier to support SQL Server in a fully clustered environment. Earlier problems with SQL Server clustering stemmed from the fact that Microsoft had to trick SQL Server into running two instances on the same node of a cluster. The registry-hacking gymnastics required to perform this trick are no longer necessary. In SQL Server 2000, cluster functionality is more stable and easier to manage than in earlier versions.

Reduction in Development Time

SQL Server 2000 includes innovations that guarantee users the greatest ease of use and the fastest time to market for their applications. The rich XML functionality in SQL Server 2000 shields Web developers from the intricacies of relational database programming; instead, they use technologies like Xpath, URL queries, and XML updategrams. Similarly, database developers are not required to learn an object-oriented language or understand all the facets of XML. The core scenario of providing XML access to an existing relational database can be accommodated with the FOR XML clause that returns XML data from a SELECT statement and the OPENXML T/SQL keyword. OpenXML provides a relational view on XML data that, with T-SQL, can be used to query data from XML, join XML data with existing relational tables, and update the database.

SQL Server enables queries to return data as XML (rather than a standard rowset) from a standard SELECT statement via the FOR XML clause. The shape of the returned XML document is determined by the XML mode specified immediately after the FOR XML clause.

Besides retrieving data as XML, it is important to be able to efficiently store data as XML, maintaining the relationships and hierarchy of data while taking full advantage of the speed offered by a high-performance database like SQL Server. SQL Server 2000 can provide an XML view of relational data as well as map XML data into relational tables. SQL Server 2000 introduces annotations (extensions) to the XDR (XML-data reduced) schema language. These annotations are used to specify a bidirectional XML-to-relational mapping. Elements and attributes in an XML document are mapped to tables and columns in the databases. By default, an element name in an annotated schema maps to a table (or view) name in the specified database and the attribute name maps to the column name. These annotations can also be used to specify the XML hierarchical relationships.

Accessing Data via Internet

SQL 2000 provides the ability to access data securely from the browser, through firewalls. It also provides fast, full-text searches of formatted documents. SQL Server 7.0 provided features to generate HTML pages from SQL Server data, to post SQL Server data to HTTP and FTP locations, and to send and receive e-mail. SQL Server 2000 delivers new and enhanced features for accessing data stored in SQL Server databases and OLAP cubes via the Web. These features ensure that data is available via intuitive user interfaces and is fully searchable, even by novice users. SQL Server 2000 enables URL-based access via several different mechanisms, all of which rely on an included ISAPI DLL that allows users to define virtual roots in Microsoft Internet Information Services (IIS) that are associated with an instance of SQL Server 2000. The URL can be specified to do the following.

  • Execute SQL queries directly. URL queries have the form http://server/vroot?sql="…" For URL queries, Internet applications can compose URL strings that reference an SQL Server 2000 virtual root and that contain a Transact-SQL statement. The Transact-SQL statement is sent to the instance of SQL Server 2000 associated with the virtual root, and the result is returned as a standard rowset. If the FOR XML clause is specified, an XML document is returned instead. Single-row, single-column queries are supported.

  • Access database objects, such as tables, directly. Direct queries have the form http://server/vroot/dbobject/xpath. For direct queries via HTTP, data is not returned as XML, enabling the direct retrieval of objects like images from the database. The Xpath syntax required in this case treats tables (and views) as elements and columns as attributes.

  • Execute template files. Template queries have the form http://server/vroot/vname?params and directly reference a template file, which is a valid XML document consisting of one or more SQL statements. When a template file is specified at the URL, the SQL commands stored in the template file are executed. The queries are replaced with the results, and the entire XML document is returned to the requestor.

  • Execute Xpath queries via an XML View. Xpath queries of XML Views have the form: http://server/vroot/vname/xpath?params Xpath queries access specific data from relational tables through XML View (annotated schema) that maps XML to relational database tables.

Web-Based Analysis

SQL Server 2000 delivers several significant features in Analysis Services that allow companies to derive additional value from their data by enabling fast, flexible analysis. Integrated data mining is new to SQL Server 2000 and is a core component for delivering a complete, end-to-end analysis solution in SQL Server. Data mining technology helps users analyze data in relational databases and multidimensional OLAP cubes to uncover patterns and trends that can be used to make predictions. The data mining capabilities in SQL Server 2000 are tightly integrated with both relational and OLAP data sources. In fact, the results of data mining can be used to create additional cube dimensions for further OLAP data analysis. And we can use them with relational databases by simply issuing SQL queries.

VI SAN Support

The VI SAN (Virtual Interface System Area Network) support in SQL Server 2000 allows SQL Server to communicate directly with devices connected via a SAN to transfer high volumes of data or transactions with low latency.

Indexed Database Views

When complex queries access views, performance can decline, as the database dynamically merges the logic needed to build the view result set into the logic needed to build the complete query result set from base table data. The overhead of this operation can be substantial, particularly for views that involve complex processing of large numbers of rows (e.g., large data aggregations or multitable joins). Because the result set is not stored permanently in the database (with a standard view), future accesses to the view are likely to incur the cost of building the result set each time the query is executed.

SQL Server 2000 enables the performance of queries accessing such views to be dramatically improved by allowing the creation of a unique clustered index on the view. When such an index is created, the view is executed and the result set is stored in the database in the same way a table with a clustered index is stored. This effectively materializes the results of the query in the database. The SQL Server query optimizer uses the view index in queries that do not directly name the view in the FROM clause. Existing queries can benefit from the improved efficiency of retrieving data from the indexed view without having to be recoded. For certain types of views, this can improve performance exponentially.

T-SQL Debugger

SQL 2000 comes equipped with a built in T-SQL debugger. The debugger supports traditional functionality such as setting breakpoints, defining watch expressions, and single-stepping through procedures. Developers can use it to view global and local variables and the call stack. It also now adds the capability for server-side statement tracing and client-side statistics to help developers tune and optimize their applications.

User-Defined Functions

In addition to the T-SQL functions built in to the product, SQL Server 2000 allows developers to create user-defined functions. User-defined functions are T-SQL functions that begin with using the CREATE FUNCTION statement. They are subroutines made up of one or more T-SQL statements and can be used to encapsulate code for reuse. They can thus save time by incorporating routinely used logic into user-defined functions.

New Data Types

SQL Server 2000 has support for three new data types: bigint, table, and sqlvariant. The data type sqlvariant is the same as the variant data type in Visual Basic. It can take any data type as an input parameter. Bigint helps in expanding data range supported by the integer data range. It supports values from –263 to 263 – 1. The table data type is for storing the temporary result sets; it works in the same way as the temp tables but is much more efficient since all calculations are done in the memory area.

Cascading DRI

Cascading declarative referential integrity (DRI) is the feature you'll most likely use. With cascading DRI, you specify a relationship between a parent and a dependent table so that a deletion or update of a row in the parent table automatically cascades to corresponding rows in the dependent table. The On Delete Cascade option has been available in Oracle for quite some time.

Distributed Partitioned Views

This feature set permits partitioning of views to horizontally partition tables across multiple servers. This feature enabled SQL Server 2000 to set its new Transaction Processing Performance Council (TPC) TPC-C benchmark of 262,243 transactions per minute (tpmC) with 12 clustered Compaq systems. Unfortunately, the current design has significant limitations and it's not easy to implement.

OLAP Services

Following are a few of the many improvements in analysis services.

  • Cube enhancements: New cube types and enhanced cube functionality have improved the scalability and the functionality of the Analysis Services.

  • Dimension enhancements: New dimension and hierarchy types, features, and improvements have improved the analysis capabilities of cubes.

  • Data mining enhancements: Data mining is new to this release and is totally integrated into online analysis.

  • Security enhancements: These include using roles on cube cells and dimension members and additional authentication methods.

  • Other enhancements: Multiuser administration, distinct count feature, MDX builder, virtual cube editor, support for Active Directory, and so on.

DTS

There have been a lot of improvements in data transformation services. I recommend Professional SQL Server 2000 DTS by Mark Chaffin, Brian Knight and Todd Robinson by Wrox Press Ltd. (2000). It's an excellent resource for anyone who is using DTS or plans to use DTS in the future.

Some of new features in DTS:

  • Lots of new tasks have been added: Dynamic Properties Task, FTP Task, MSMQ, Execute Package Task, transfer logins, and so on.

  • Enhanced logging facilities.

  • You can now save DTS Packages to VB files. It is an excellent way to learn how to program DTS packages in VB. Even in SQL Server 7.0, you could script out a DTS package using the ScrptPkg utility, but it's much easier with SQL 2000.

  • Multiphase data pump

  • Using parameterized queries

  • Using gobal variables to pass information between DTS packages

INSTEAD OF and AFTER Triggers

INSTEAD OF triggers are executed instead of a triggering action (for example, INSERT, UPDATE, DELETE). They can also be defined on views, in which case they greatly extend the types of updates a view can support. AFTER triggers fire after the triggering action. SQL Server 2000 introduces the ability to specify which AFTER triggers fire first and last.

Collation Enhancements

SQL Server 2000 replaces code pages and sort orders with collations. SQL Server 2000 includes support for most collations supported in earlier versions of SQL Server and introduces a new set of collations based on Windows collations. Collations can be now be specified at the database level or at the column level. Previously, code pages and sort orders could be specified only at the server level and applied to all databases on a server. Collations support code page translations. Operations with char and varchar operands having different code pages are now supported. Code page translations are not supported for text operands. ALTER DATABASE can be used to change the default collation of a database.

Full-Text Search Enhancements

Full-text search now includes change tracking and image filtering. Change tracking maintains a log of all changes to the full-text indexed data. You can update the full-text index with these changes by flushing the log manually, on a schedule, or as they occur, using the background update index option. Image filtering allows you to index and query documents stored in image columns. The user provides the document type in a column that contains the file name extension that the document would have had if it were stored as a file in the file system. Using this information, full-text search is able to load the appropriate document filter to extract textual information for indexing.

Multiple Instances of SQL Server

SQL Server 2000 supports running multiple instances of the relational database engine on the same computer. Each computer can run one instance of the relational database engine from SQL Server version 6.5 or 7.0 along with one or more instances of the database engine from SQL Server 2000. Each instance has its own set of system and user databases. Applications can connect to each instance on a computer similar to the way they connect to instances of SQL Servers running on different computers. The SQL Server 2000 utilities and administration tools have been enhanced to work with multiple instances.

Index Enhancements

Indexes can now be created on computed columns. You can specify whether indexes are built in ascending or descending order and whether the database engine should use parallel scanning and sorting during index creation. The CREATE INDEX statement can now use the tempdb database as a work area for the sorts required to build an index. This results in improved disk read and write patterns for the index creation step and makes it more likely that index pages will be allocated in contiguous strips. In addition, not only the initial table scan but the complete process of creating an index is eligible for parallel operations.

Failover Clustering Enhancements

The administration of failover clusters has been greatly improved to make it very easy to install, configure, and maintain a Microsoft SQL Server 2000 failover cluster. Additional enhancements include the ability to failover and failback to or from any node in a SQL Server 2000 cluster, the ability to add or remove a node from the cluster through SQL Server 2000 Setup, and the ability to reinstall or rebuild a cluster instance on any node in the cluster without affecting the other cluster node instances. The SQL Server 2000 utilities and administration tools have been enhanced to work with failover clusters.

Net-Library Enhancements

The SQL Server 2000 Net-Libraries have been rewritten to virtually eliminate the need to administer Net-Library configurations on client computers when connecting SQL Server 2000 clients to instances of SQL Server 2000. The new Net-Libraries also support connections to multiple instances of SQL Server on the same computer and support Secure Sockets Layer encryption over all Net-Libraries. SQL Server 2000 introduces Net-Library support for Virtual Interface Architecture (VIA) system-area networks that provide high-speed connectivity between servers, such as between application servers and database servers.

64-GB Memory Support

The Microsoft SQL Server 2000 Enterprise Edition can use the Microsoft Windows 2000 Advanced Windows Extension (AWE) API to support up to 64 GB of physical memory (RAM) on a computer.

Distributed Query Enhancements

SQL Server 2000 introduces a new OPENDATASOURCE function, which you can use to specify ad hoc connection information in a distributed query. SQL Server 2000 also specifies methods that OLE DB providers can use to report the level of SQL syntax supported by the provider and statistics on the distribution of key values in the data source. The distributed query optimizer can then use this information to reduce the amount of data that has to be sent from the OLE DB data source. SQL Server 2000 delegates more SQL operations to OLE DB data sources than earlier versions of SQL Server. Distributed queries also support the other functions introduced in SQL Server 2000, such as multiple instances, mixing columns with different collations in result sets, and the new bigint and sql_variant data types. SQL Server 2000 distributed queries add support for the OLE DB Provider for Exchange and the Microsoft OLE DB Provider for Microsoft Directory Services.

Kerberos and Security Delegation

SQL Server 2000 uses Kerberos to support mutual authentication between the client and the server, as well as the ability to pass the security credentials of a client between computers, so that work on a remote server can proceed using the credentials of the impersonated client. With Microsoft Windows 2000, SQL Server 2000 uses Kerberos and delegation to support both integrated authentication and SQL Server logins.

Backup and Restore Enhancements

SQL Server 2000 introduces a new, more easily understood model for specifying backup and restore options. The new model makes it clearer that you are balancing increased or decreased exposure to losing work against the performance and log space requirements of different plans. SQL Server 2000 introduces support for recovery to specific points of work using named log marks in the transaction log and the ability to do partial database restores. Users can define passwords for backup sets and media sets that prevent unauthorized users from accessing SQL Server backups.

Scalability Enhancements for Utility Operations

SQL Server 2000 enhancements for utility operations include faster differential backups, parallel database console command (DBCC) checking, and parallel scanning. Differential backups can now be completed in a time that is proportional to the amount of data changed since the last full backup. DBCC can be run without taking shared table locks while scanning tables, thereby enabling them to be run concurrently with update activity on tables. Additionally, DBCC now takes advantage of multiple processors, thus enabling near-linear gain in performance in relation to the number of CPUs (provided that I/O is not a bottleneck).

Text in Row Data

SQL Server 2000 supports a new text in row table option that specifies that small text, ntext, and image values be placed directly in the data row instead of in a separate page. This reduces the amount of space used to store small text, ntext, and image data values and reduces the amount of disk I/O needed to process these values.


Summary

We can clearly conclude from this discussion that SQL 2000 has lot more features/enhancements than does SQL Server 7.0, especially in the area of Web development. To summarize, here are the key improvements SQL 2000.

  • Fully Web-enabled

  • Highly scalable and more reliable

  • More extensive feature set that reduces development effort and maintenance time

  • Web-based analysis that helps users analyze data in relational databases and multidimensional OLAP cubes to uncover patterns and trends that can be used to make predictions.

  • Indexed database views, multiple instance support, and reduction in application development time are other important factors that will prove to be very beneficial for the projects.

  • More robust data transformation services that help a great deal during the migration of databases from heterogeneous systems to SQL Server as well as aiding easy development of administration utilities.


Chapter 3. XML Support in SQL Server 2000

Extensible Markup Language (XML) is used to describe the contents of a set of data and how the data should be output to a device or displayed on a Web page. Markup languages originated as ways for publishers to indicate to printers how the content of a newspaper, magazine, or book should be organized. Markup languages for electronic data perform the same function for electronic documents that can be displayed on different types of electronic gear.

Both XML and the Hypertext Markup Language (HTML) are derived from Standard Generalized Markup Language (SGML). SGML is a very large, complex language that is difficult to fully use for publishing data on the Web. HTML is a simpler, more specialized markup language than SGML but has a number of limitations when working with data on the Web. XML is smaller than SGML and more robust than HTML, so it is becoming an increasingly important language in the exchange of electronic data through the Web or intracompany networks.

In a relational database such as Microsoft SQL Server 2000 or Oracle, all operations on the tables in the database produce a result in the form of a table. The result set of a SELECT statement is in the form of a table. Traditional client/server applications that execute a SELECT statement process the results by fetching one row or block of rows from the tabular result set at a time and mapping the column values into program variables. Web application programmers, on the other hand, are more familiar with working with hierarchical representations of data in XML or HTML documents.

SQL Server 2000 introduces support for XML with the following features:

  • Ability to access SQL Server through a URL

  • Support for XML-Data schemas and the ability to specify Xpath queries against these schemas

  • Ability to retrieve XML data using the SELECT statement and the FOR XML clause and to write XML data using the OpenXML rowset provider

  • Enhancements to the Microsoft SQL Server 2000 OLE DB provider (SQLOLEDB) that allow XML documents to be set as command text and to return result sets as a stream

Let's look at SQL Server 2000's XML enhancements and an example of how you can XML-enable your existing systems by extracting data from SQL Server 2000 as XML, formatting the data the way you want, then using the data in your applications.


Generating XML with the SELECT Statement

It is possible to retrieve XML formatted results using the Transact-SQL SELECT statement and the FOR XML clause. Following are the modes within the FOR XML clause:

  • FOR XML RAW. Returns each row of the query result set in a generic row element with each nonnull column as an attribute.

  • FOR XML AUTO. Returns the query results as nested elements corresponding to the tables in the FROM clause of the select statement. By default, the AUTO mode maps the selected table columns as attributes of the element; however, if the ELEMENTS option is enabled, the selected table columns become subelements of the table element.

  • FOR XML EXPLICIT. Allows the query more control over the formatting of the returned XML result set. The query is designed to explicitly define the XML document returned; therefore, the author of the query becomes responsible for creating a well-formed XML document. This option allows for the most control over the query result set.

To generate an XML formatted result set, start SQL Query Analyzer, and then connect to the Northwind database. Execute the following query:

SELECT * from CUSTOMERS FOR XML RAW

SQL Server returns the result set in a single column named in part with a globally unique identifier (GUID). Unlike the XML Technology preview, the results are not wrapped in the <root></root> tags that are needed for a properly formatted XML document, permitting multiple queries to produce one XML document.


Generating XML over the Internet

To exploit the full functionality of SQL Server XML support over the Internet, download and install the latest Web release for SQL Server 2000 XML support from the Developer Center on MSDN. Two Web releases have been made so far.

Before accessing SQL Server over the Internet, a virtual directory must be set up on the computer running Microsoft Internet Information Services (IIS) specifically for the XML queries. Following are the steps for enabling IIS for SQL Server XML support.

  1. On the Start menu, point to Program Files, click Microsoft SQL Server, and then click "Configure SQL XML support in IIS."

  2. Expand your server, right-click the Default Web Site folder, point to New, and then click Virtual Directory.

  3. On the General tab of the Properties page, type XMLTest for the virtual directory name.

  4. For Local Path, type C:\ reskit\xmltest.

  5. On the Security tab, type a user name and password with administrative permission in the Northwind database.

  6. On the Data Source tab, type the database server name, and then type Northwind as the database name.

  7. On the Settings tab, select Virtual URL Queries and Template Queries.

  8. On the Virtual Names tab, click New, and then type a new virtual name of queries with a type of template and a path to C:\reskit\xmltest.

  9. Click OK to save your settings, and then close the IIS Virtual Directory Manager for SQL Server.


Retrieving XML-Formatted Data from SQL Server

You can query SQL Server 2000 and retrieve an XML formatted document in either of the following ways: (1) Send SQL Server a query string (either a Transact-SQL SELECT statement or an Xpath query) in the HTTP request. Important security considerations arise when a database is exposed to queries through the browser. Any user can execute a query within the user context of the username specified in the virtual root. If you choose to allow this type of access, make sure that you apply the appropriate permissions within the database for this user. (2) Create template files that contain Transact-SQL or Xpath queries and attach XML Template (.xsl) files to format the XML document that is returned. (This is generally easier and more secure than sending a query string.)

The following steps are involved in retrieving XML using a Transact-SQL statement in a URL.

  1. In the Microsoft Internet Explorer address bar, type the following query without line breaks.

    http://localhost/xmltest?sql=SELECT+contactname,+phone+FROM+Customers+FOR+XML+raw&r
oot=root

  2. With your cursor still in the address bar, press ENTER to execute the query. The following results have been truncated for readability.

<root>
<row contactname="Maria Anders" phone="030-0074321" />
<row contactname="Ana Trujillo" phone="(5) 555-4729" />
<row contactname="Antonio Moreno" phone="(5) 555-3932" />
<row contactname="Thomas Hardy" phone="(171) 555-7788" />
<row contactname="Christina Berglund" phone="0921-12 34 65" />
<row contactname="Hanna Moos" phone="0621-08460" />
     ...
</root>

The contactname and phone columns named in the query are attributes of the row element. By using a JOIN and the FOR XML AUTO clause, the results appear as XML elements to form a more hierarchical structure.

To execute a query that generates AUTO formatted XML results, follow these steps.

  1. In Microsoft Internet Explorer, copy the following query into the address bar without line breaks.

    http://localhost/xmltest?sql=SELECT+customers.CustomerID,OrderID,OrderDate+FROM+Cus
tomers+INNER+JOIN+Orders+ON+customers.CustomerID=orders.CustomerID+ORDER+BY+custome
rs.CustomerID,OrderID+FOR+XML+AUTO&root=root

  2. With your cursor still in the address bar, press ENTER to execute the query. The following results have been truncated for readability.

- <root>
- <Customers CustomerID="ALFKI">
<Orders OrderID="10643" OrderDate="1997-08-25T00:00:00" />
<Orders OrderID="10692" OrderDate="1997-10-03T00:00:00" />
<Orders OrderID="10702" OrderDate="1997-10-13T00:00:00" />
<Orders OrderID="10835" OrderDate="1998-01-15T00:00:00" />
<Orders OrderID="10952" OrderDate="1998-03-16T00:00:00" />
<Orders OrderID="11011" OrderDate="1998-04-09T00:00:00" />
</Customers>
- <Customers CustomerID="ANATR">
<Orders OrderID="10308" OrderDate="1996-09-18T00:00:00" />
<Orders OrderID="10625" OrderDate="1997-08-08T00:00:00" />
<Orders OrderID="10759" OrderDate="1997-11-28T00:00:00" />
<Orders OrderID="10926" OrderDate="1998-03-04T00:00:00" />
</Customers>
...
</root>

You can expand or collapse customer information to view the Orders elements of a particular customer.


Reserved Characters

Both URLs and XML documents have a set of reserved characters. Table 3.1 lists each special character, its function, and the hexadecimal value that can be used to replace the character. Queries in a URL must replace the special characters that are used in URLs with their hex values. Queries in XML Template documents must properly encode the special characters used in XML with the values listed in Table 3.2.

To execute an XML-generating Transact-SQL query with special characters, do the following.

  1. In Microsoft Internet Explorer, enter the following query into the address bar without line breaks.

    http://localhost/xmltest?sql=SELECT+'<ROOT>';SELECT+DISTINCT+ContactTitle+FROM+Cust
omers+WHERE+ContactTitle+LIKE+'Sa%25'+ORDER+BY+ContactTitle+FOR+XML+AUTO;SELECT+'</
ROOT>'

    Table 3.1. Special characters in URLs
    Character Use Hex value
    + Indicates a space %20
    / Separates directories and subdirectories %2f
    ? Separates the URL and the parameters %3f
    % Specifies special characters %25
    # Indicates bookmarks %23
    & Separates URL parameters %26

  2. With your cursor still in the address bar, press ENTER to execute the query.

In addition to the use of special characters, this query also manually generates the <ROOT></ROOT> tags by explicitly naming them.

Stored procedures can also be executed from the URL. They can be called either with the Transact-SQL syntax EXECUTE sp_who or with the ODBC syntax {call sp_who} . The following steps show how a stored procedure is created to dynamically publish the Northwind product catalog to third parties. To execute a stored procedure using Internet Explorer, do the following.

  1. Start SQL Query Analyzer, and then connect to your server.

  2. Ensure that the current Query window is pointing to the Northwind database, and then execute the following:

    CREATE PROCEDURE USP_xmlCatalog
as
SELECT
     Category.CategoryName as 'Category',
     product.productName as 'Product',
     Product.UnitPrice as 'Price'
FROM
     Categories Category
     INNER JOIN Products Product
     ON Category.CategoryID = Product.CategoryID
Order by
     Category.CategoryName,
     Product.ProductName
FOR XML AUTO

    Table 3.2. Special characters in XML
    Special character Use Entity encoding
    < Begins a tag &gt;
    > Ends a tag &lt;
    " Quotation mark &quot;
    ' Apostrophe &apos;
    & Ampersand &amp;

  3. In Microsoft Internet Explorer, enter the following query into the address bar without line breaks.

    http://localhost/xmltest?sql= EXECUTE+USP_XMLCatalog&root=root

  4. With your cursor still in the address bar, press ENTER to execute the query.

A list of products in the Northwind database will appear, sorted by category. The following results are truncated to enhance readability.

<?xml version="1.0" encoding="utf-8" ?>
- <root>
- <Category Category="Beverages">
<Product Product="Chai" Price="80" />
<Product Product="Chang" Price="19" />
<Product Product="Chartreuse verte" Price="80" />
<Product Product="Côte de Blaye" Price="263.5" />
<Product Product="Guaraná Fantástica" Price="4.5" />
<Product Product="Ipoh Coffee" Price="46" />
<Product Product="Lakkalikööri" Price="80" />
<Product Product="Laughing Lumberjack Lager" Price="14" />
<Product Product="Outback Lager" Price="15" />
<Product Product="Rhönbräu Klosterbier" Price="7.75" />
<Product Product="Sasquatch Ale" Price="14" />
<Product Product="Steeleye Stout" Price="80" />
</Category>
       - <Category>
       ...
       </Category>
       ...
</root>

This stored procedure can be modified to take a parameter to allow for queries against the database for specific categories. A default parameter, USP_xmlCatalog, can be added to the stored procedure. Next, the procedure can be called from Internet Explorer.

  1. Execute the following code in SQL Query Analyzer.

    ALTER PROCEDURE sp_rk_xmlCatalog
      @CategoryName nvarchar(30) = '%'
as
SELECT
      Category.CategoryName as 'Category',
      product.productName as 'Product',
      Product.UnitPrice as 'Price'
FROM
      Categories Category
      INNER JOIN Products Product
      ON Category.CategoryID = Product.CategoryID
WHERE
      Category.CategoryName like @CategoryName
Order by
      Category.CategoryName,
      Product.ProductName
FOR XML AUTO

  2. In Microsoft Internet Explorer, enter the following query into the address bar, and then press ENTER.

http://localhost/xmltest/?sql=EXECUTE+sp_rk_xmlCatalog+@CategoryName="Confections"&root=root

Only the products in the Confections category should be listed.

- <root>
- <Category Category="Confections">
               <Product Product="Chocolade" Price="12.75" />
               <Product Product="Gumbär Gummibärchen" Price="31.23" />
               <Product Product="Maxilaku" Price="20" />
               <Product Product="NuNuCa Nuß-Nougat-Creme" Price="14" />
               <Product Product="Pavlova" Price="17.45" />
               <Product Product="Schoggi Schokolade" Price="43.9" />
               <Product Product="Scottish Longbreads" Price="12.5" />
               <Product Product="Sir Rodney's Marmalade" Price="81" />
               <Product Product="Sir Rodney's Scones" Price="10" />
<Product Product="Tarte au sucre" Price="49.3" />
<Product Product="Teatime Chocolate Biscuits" Price="9.2" />
<Product Product="Valkoinen suklaa" Price="16.25" />
<Product Product="Zaanse koeken" Price="9.5" />
  </Category>
</root>


XML Templates

As queries grow in complexity, it becomes impractical to send them through URLs. Furthermore, you might not want to expose the queries in the address bar of the browser where they can be edited by third parties. Instead, queries can be stored in XML templates. Templates are well-formed XML documents that contain one or more SQL statements or Xpath queries. The XML virtual root for IIS must be specifically configured to use templates before they will work on your system.

To enable the XML virtual root within IIS to use templates, create a virtual name within the Virtual Directory that is enabled for XML templates. To allow IIS virtual roots to use template files, take the following steps.

  1. On the Start menu, point to Programs, click Microsoft SQL Server, and then click Configure SQL XML Support in IIS.

  2. Expand your server, and then click Default Web Site.

  3. Right-click the XMLTest virtual directory, and then click Properties.

  4. On the Settings tab, ensure that the Allow template queries check box is selected.

  5. On the Virtual Names tab, click New to add a virtual name for the templates folder.

  6. Use a virtual name for templates that is type Template, and then set the path to C:\reskit\xmltest.

  7. Click Save, and then close the IIS Virtual Directory Manager for SQL Server.

The following steps show how to create a simple XML template that executes a simple Transact-SQL query and access the result set using Internet Explorer.

  1. Paste the following code into your text editor:

    <ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql">
<sql:query>
SELECT
      SupplierID,
      CompanyName,
      ContactName,
      Phone
FROM
      suppliers
ORDER BY
      CompanyName
FOR XML AUTO
</sql:query>
</ROOT>

  2. Save the file as C:\xmltest\suppliers.xml.

  3. In Microsoft Internet Explorer, enter the following query into the address bar, and then press ENTER.

http://localhost/xmltest/templates/suppliers.xml

The URL includes the \templates directory, even though the template is stored directly in the xmltest directory. This is because it is mapped to the suppliers.xml template through the template's virtual name that was created earlier.


Using EXPLICIT Mode

EXPLICIT mode brings an additional level of complexity to a query, but it allows for much more control over the XML document that is generated by the query. EXPLICIT mode requires that the query produce a result set in the universal table format. The universal table fully describes the resulting XML document.

Table 3.3 demonstrates the logical structuring of the data within a query that returns products, categories, and product details such as price and supplier.

The column names are encoded using XML generic identifiers and attribute names. These names are specified as

ElementName!TagNumber!AttributeName!Directive

The following steps show how to return the Northwind store catalog using EXPLICIT mode.

  1. Paste the following code into your text editor:

    SELECT
      1         as 'Tag',
      null      as 'Parent',
      Category.CategoryName as 'Category!1!Name',
      null      as 'Product!2!Name',
      null      as 'ProductDetail!3!Supplier',
      null      as 'ProductDetail!3!Price',
      null      as 'ProductDetail!3!NumberSold'
FROM
      categories Category

UNION ALL
SELECT
      2,
      1,
      Category.CategoryName,
      Product.ProductName,
      null,
      null,
      null
FROM
      categories Category
      INNER JOIN products Product
      on Category.CategoryID = Product.CategoryID
UNION ALL
SELECT
      3,
      2,
      Category.CategoryName,
      Product.ProductName,
      null,
      Product.UnitPrice,
      null
FROM
      categories Category
      INNER JOIN products Product
      on Category.CategoryID = Product.CategoryID
      INNER JOIN suppliers s
      on product.supplierid = s.supplierid

UNION ALL
SELECT
      3,
      2,
      Category.CategoryName,
      Product.ProductName,
      s.CompanyName,
      null,
      null
FROM
      categories Category
      INNER JOIN products Product
      on Category.CategoryID = Product.CategoryID
      INNER JOIN suppliers s
      on product.supplierid = s.supplierid

UNION ALL
SELECT
      3,
      2,
      Category.CategoryName,
      Product.ProductName,
      null,
      null,
      sum(quantity)
FROM
      categories Category
INNER JOIN products Product
      on Category.CategoryID = Product.CategoryID
      INNER JOIN suppliers s
      on product.supplierid = s.supplierid
      LEFT JOIN [order details] od
      ON product.productid = od.productid
group by category.categoryname,product.productname,s.companyname
ORDER BY
      'Category!1!Name',
      'Product!2!Name',
      'ProductDetail!3!Supplier',
      'ProductDetail!3!Price',
      'ProductDetail!3!NumberSold'
FOR XML EXPLICIT

    Table 3.3.
    Tag | Parent | Category!1 !Name | Product!2 !Name | Product!2 !Supplier!Element | Product!2 !Price!Element
    1 | NULL | Beverages | NULL | NULL | NULL
    2 | 1 | NULL | Chai | Exotic liquids | 80
    2 | 1 | NULL | Chang | Exotic liquids | 19
    1 | NULL | Condiments | NULL | NULL | NULL
    2 | 1 | NULL | Aniseed syrup | Exotic liquids | 10

  2. Save the file as Catalog.xml.

  3. In Microsoft Internet Explorer, enter the following query into the address bar, and then press ENTER.

http://localhost/xmltest/templates/catalog.xml

The results are hierarchical; each piece of information is stored as an XML element instead of an attribute to an element. Elements on the same level, such as those owned by the Product Element, can have different attributes, such as Price and Supplier.

- <ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql">
- <Category Name="Beverages">
- <Product Name="Chai">
                  <ProductDetail Price="80" />
                  <ProductDetail Supplier="Exotic Liquids" />
            </Product>
- <Product Name="Chang">
                  <ProductDetail Price="19" />
                  <ProductDetail Supplier="Exotic Liquids" />
            </Product>
            ...
      </Category>
      ...
</root>


XSL Stylesheets

If the XML formatted information will be displayed directly to the user, you can use Extensible Stylesheet Language (XSL) stylesheets to format the information. You can also use XSL to filter the XML document so that only certain information is displayed. For example, if you are publishing a list of all of your employees, you might want to display their phone numbers internally but not externally to the public. The following steps create an XML template that returns employee information and two XSL stylesheets, one to format information for external use and another for internal use.

  1. Save the following XML template as EmpList.xml.

    <?xml version ='1.0' encoding='UTF-8'?>
 <root xmlns:sql='urn:schemas-microsoft-com:xml-sql'>
   <sql:query>
      SELECT FirstName, LastName, Extension 'Extension' FROM Employees FOR XML AUTO
   </sql:query>
</root>

  2. View the data from the XML template in Internet Explorer by loading http://localhost/xmltest/templates/emplist.xml

The attributes in the Employee Element are FirstName, LastName, and Extension.

The next example creates a stylesheet to format the results into an HTML document. The stylesheet performs recursion on the returned XML document for all Employee elements and creates a new row in an HTML table to store the attributes.

  1. Save the following XSL stylesheet in your XMLtest directory as empInternal.xsl.

    <?xml version='1.0' encoding='UTF-8'?>
 <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

    <xsl:template match = '*'>
        <xsl:apply-templates />
    </xsl:template>
    <xsl:template match = 'Employees'>
       <TR>
         <TD><xsl:value-of select = '@FirstName' /></TD>
         <TD><B><xsl:value-of select = '@LastName' /></B></TD>
         <TD><xsl:value-of select = '@Extension' /></TD>
       </TR>
    </xsl:template>
    <xsl:template match = '/'>
      <HTML>
        <HEAD>
           <STYLE>th { background-color: #CCCCCC } </STYLE>
        </HEAD>
        <BODY>
         <TABLE border='1' style='width:300;'>
           <TR><TH colspan='3'>Employees</TH></TR>
           <TR><TH >First name</TH><TH>Last name</TH><TH>Extension</TH></TR>
           <xsl:apply-templates select = 'root' />
         </TABLE>
        </BODY>
      </HTML>
    </xsl:template>
</xsl:stylesheet>

  2. View the formatted results of the XML template in Internet Explorer by loading http://localhost/xmltest/templates/emplist.xml?xsl=empInternal.xsl

The stylesheet is applied to the XML template from the URL. Different versions of this Web page can be displayed to different types of users by applying different stylesheets to the same XML.

The next example creates an XSL stylesheet to post employee names externally, without the phone numbers.

  1. Save the following stylesheet as EmpInternal.xsl.

    <?xml version='1.0' encoding='UTF-8'?>
 <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:template match = '*'>
        <xsl:apply-templates />
    </xsl:template>
    <xsl:template match = 'Employees'>
       <TR>
         <TD><xsl:value-of select = '@FirstName' /></TD>
         <TD><B><xsl:value-of select = '@LastName' /></B></TD>
       </TR>
    </xsl:template>
    <xsl:template match = '/'>
      <HTML>
        <HEAD>
           <STYLE>th { background-color: #CCCCCC } </STYLE>
        </HEAD>
        <BODY>
         <TABLE border='1' style='width:300;'>
           <TR><TH colspan='3'>Employees</TH></TR>
           <TR><TH >First name</TH><TH>Last name</TH></TR>
           <xsl:apply-templates select = 'root' />
         </TABLE>
        </BODY>
      </HTML>
    </xsl:template>
</xsl:stylesheet>

  2. View the new results in Internet Explorer by loading

http://localhost/xmltest/templates/emplist.xml?xsl=empExternal.xsl.


XML System Stored Procedures

Microsoft SQL Server 2000 provides these system stored procedures that are used in conjunction with OPENXML:

  • sp_xml_preparedocument

  • sp_xml_removedocument

To write queries using OPENXML, you must first create an internal representation of the XML document by calling sp_xml_preparedocument. The stored procedure returns a handle to the internal representation of the XML document. This handle is then passed to OPENXML, which provides rowset views of the document based on Xpaths; namely one row pattern and one or more column patterns.

The internal representation of an XML document can be removed from memory by calling sp_xml_removedocument system stored procedure.


OPENXML

OPENXML provides a rowset view over an XML document. Because OPENXML is a rowset provider, it can be used in Transact-SQL statements in which rowset providers such as a table, a view, or the OPENROWSET function can appear.

Syntax

  • OPENXML(idoc int [in],rowpattern nvarchar[in], [flags byte[in]]) [WITH (SchemaDeclaration | TableName)]

Arguments

  • idoc is the document handle of the internal representation of an XML document. The internal representation of an XML document is created by calling sp_xml_preparedocument.

  • rowpattern is the Xpath pattern used to identify the nodes (in the XML document whose handle is passed in the idoc parameter) to be processed as rows.

  • flags indicates the mapping that should be used between the XML data and the relational rowset and how the spill over column should be filled. flags is an optional input parameter and can be one of the values shown in Table 3.4.

  • SchemaDeclaration is the schema definition of the form./

  • ColName ColType [ColPattern | MetaProperty][, ColName ColType [ColPattern | MetaProperty]…]

  • ColName is the column name in the rowset.

  • ColType is the SQL data type of the column in the rowset. If the column types differ from the underlying XML data type of the attribute, type coercion occurs. If the column is of type timestamp, the present value in the XML document is disregarded when selecting from an OPENXML rowset, and the autofill values are returned.

  • ColPattern is an optional, general Xpath pattern that describes how the XML nodes should be mapped to the columns. If the ColPattern is not specified, the default mapping (attribute-centric or element-centric mapping as specified by flags) takes place. ColPattern is used to specify the special nature of the mapping (in case of attribute-centric and element-centric mapping) that overwrites or enhances the default mapping indicated by flags. The Xpath patterns specified as ColPattern also supports the metaproperties.

  • MetaProperty is one of the metaproperties provided by OPENXML. If the metaproperty is specified, the column contains information provided by the metaproperty. The metaproperties allow you to extract more information (such as relative position and namespace information) about XML nodes than is visible in the textual representation.

  • TableName is the name that can be given (instead of SchemaDeclaration) if a table with the desired schema already exists and no column patterns are required. The WITH clause provides a rowset format (and additional mapping information as necessary) using either SchemaDeclaration or specifying an existing TableName. If the optional WITH clause is not specified, the results are returned in an edge table format. Edge tables represent the fine-grained XML document structure (e.g. element/attribute names, the document hierarchy, the namespaces, PIs and so on) in a single table (Table 3.5).

Table 3.4. Byte values
Byte value | Description
0 | Defaults to attribute-centric mapping.
1 | Use the attribute-centric mapping. Can be combined with XML_ELEMENTS, in which case, attribute-centric mapping is applied first, and then element-centric mapping is applied for all columns not yet dealt with.
2 | Use the element-centric mapping. Can be combined with XML_ATTRIBUTES, in which case, attribute-centric mapping is applied first, and then element-centric mapping is applied for all columns not yet dealt with.
8 | Can be combined (logical OR) with XML_ATTRIBUTES or XML_ELEMENTS. In context of retrieval, this flag indicates that the consumed data should not be copied to the overflow property @mp:xmltext.

Using a simple SELECT statement with OPENXML creates an internal representation of the XML image using sp_xml_preparedocument. A SELECT statement using an OPENXML rowset provider is then executed against the internal representation of the XML document. The flag value is set to 1 indicating attribute-centric mapping. Therefore, the XML attributes map to the columns in the rowset. The rowpattern specified as /ROOT/Customers identifies the <Customers> nodes to be processed. The optional ColPattern (column pattern) is not specified because the column name matches the XML attribute names.

Table 3.5. Structure of the edge table
Column name | Data type | Description
id | bigint | Unique ID of the document node. The root element has an ID value 0. The negative ID values are reserved.
parentid | bigint | Parent of the node. The parent identified by this ID is not necessarily the parent element, but whose parent is identified by this ID depends on the NodeType of the node. For example, if the node is a text node, the parent of it may be an attribute node. If the node is at the top level in the XML document, its ParentID is NULL.
nodetype | int | Node type; integer that corresponds to the XML DOM node type numbering (see DOM for node information). The node types are 1= Element node 2 = Attribute node 3 = Text node.
localname | nvarchar | Local name of the element or attribute; NULL if the DOM object does not have a name.
prefix | nvarchar | Namespace prefix of the node name.
namespaceuri | nvarchar | Namespace URI of the node. If the value is NULL, no namespace is present.
datatype | nvarchar | Actual data type of the element or attribute row and is NULL otherwise. The data type is inferred from the inline DTD or from the inline schema.
prev | bigint | XML ID of the previous sibling element. Is NULL if there is no direct previous sibling.
text | ntext | Contains the attribute value or the element content in text form (or is NULL if the edge table entry does not need a value).

The OPENXML rowset provider creates a two-column rowset (CustomerID and ContactName) from which the SELECT statement retrieves the necessary columns (in this case, all the columns).

DECLARE @idoc int
DECLARE @doc varchar(1000)
SET @doc ='
<ROOT>
<Customer CustomerID="VINET" ContactName="Paul Henriot">
   <Order CustomerID="VINET" EmployeeID="5" OrderDate="1996-07-04T00:00:00">
      <OrderDetail OrderID="10248" ProductID="11" Quantity="12"/>
      <OrderDetail OrderID="10248" ProductID="42" Quantity="10"/>
   </Order>
</Customer>
<Customer CustomerID="LILAS" ContactName="Carlos Gonzlez">
   <Order CustomerID="LILAS" EmployeeID="3" OrderDate="1996-08-16T00:00:00">
      <OrderDetail OrderID="10283" ProductID="72" Quantity="3"/>
   </Order>
</Customer>
</ROOT>'
-- Create an internal representation of the XML document.
EXEC sp_xml_preparedocument @idoc OUTPUT, @doc
-- Execute a SELECT statement that uses the OPENXML rowset provider.
SELECT    *
FROM       OPENXML (@idoc, '/ROOT/Customer',1)
            WITH (CustomerID  varchar(10),
                  ContactName varchar(20))

Here is the result set:

CustomerID  ContactName
----------  -----------
VINET       Paul Henriot
LILAS       Carlos Gonzlez

If the same SELECT statement is executed with flags set to 2, indicating element-centric mapping, the values of CustomerID and ContactName for both customers in the XML document are returned as NULL, because the <Customers> elements do not have any subelements.

Here is the result set:

CustomerID  ContactName
----------  -----------
NULL        NULL
NULL        NULL

Specifing ColPattern for mapping between columns and the XML attributes returns customer ID, order date, product ID, and quantity attributes from the XML document. The rowpattern identifies the <OrderDetails> elements. ProductID and Quantity are attributes of the <OrderDetails> element. However, OrderID, CustomerID, and OrderDate are the attributes of parent element (<Orders>).

The optional ColPattern is specified, indicating that the OrderID, CustomerID, and OrderDate in the rowset map to the attributes of the parent of the nodes identified by rowpattern in the XML document. The ProdID column in the rowset maps to the ProductID attribute, and the Qty column in the rowset maps to the Quantity attribute of the nodes identified in rowpattern.

Although the element-centric mapping is specified by the flags parameter, the mapping specified in ColPattern overwrites this mapping.

declare @idoc int
declare @doc varchar(1000)
set @doc ='
<ROOT>
<Customer CustomerID="VINET" ContactName="Paul Henriot">
   <Order OrderID="10248" CustomerID="VINET" EmployeeID="5"
           OrderDate="1996-07-04T00:00:00">
      <OrderDetail ProductID="11" Quantity="12"/>
      <OrderDetail ProductID="42" Quantity="10"/>
   </Order>
</Customer>
<Customer CustomerID="LILAS" ContactName="Carlos Gonzlez">
   <Order OrderID="10283" CustomerID="LILAS" EmployeeID="3"
           OrderDate="1996-08-16T00:00:00">
      <OrderDetail ProductID="72" Quantity="3"/>
   </Order>
</Customer>
</ROOT>'
-- Create an internal representation of the XML document.
exec sp_xml_preparedocument @idoc OUTPUT, @doc
-- SELECT stmt using OPENXML rowset provider
SELECT *
FROM   OPENXML (@idoc, '/ROOT/Customer/Order/OrderDetail',2)
         WITH (OrderID       int         '../@OrderID',
               CustomerID  varchar(10) '../@CustomerID',
               OrderDate   datetime    '../@OrderDate',
               ProdID      int         '@ProductID',
               Qty         int         '@Quantity')

This is the result:

OrderID  CustomerID  OrderDate                 ProdID  Qty
-------  ----------  -----------------------   ------  ---
10248    VINET       1996-07-04 00:00:00.000   11      12
10248    VINET       1996-07-04 00:00:00.000   42      10
10283    LILAS       1996-08-16 00:00:00.000   72      3


Writing Queries Against an XML Document

To write queries against an XML document using OPENXML, you must first call sp_xml_preparedocument, which parses the XML document and returns a handle to the parsed document that is ready for consumption. The parsed document is a tree representation of various nodes (elements, attributes, text, comments, and so on) in the XML document. The document handle is passed to OPENXML, which then provides a rowset view of the document based on the parameters passed to it.

The internal representation of an XML document must be removed from memory by calling sp_xml_removedocument system stored procedure to free the memory.

OPENXML Parameters

Following are the parameters of OPENXML.

  • Xpath expression to identify the nodes to be mapped to rows (rowpattern)

  • Description of the rowset to be generated

  • Mapping between the rowset columns and the XML nodes

  • XML document handle (idoc). The document handle is returned by the sp_xml_preparedocument stored procedure.

Xpath Expression to Identify the Nodes to Be Processed (rowpattern)

The Xpath expression specified as rowpattern identifies a set of nodes in the XML document. Each node identified by rowpattern corresponds to a single row in the rowset generated by OPENXML.

The nodes identified by the Xpath expression can be any XML node (elements, attributes, processing instructions, and so on) in the XML document. If rowpattern identifies a set of elements in the XML document, there is one row in the rowset for each element node identified. For example, if rowpattern ends in an attribute, a row is created for each attribute node selected by rowpattern.

Description of the Rowset to Be Generated

A rowset schema must be provided to OPENXML to generate the rowset. The following options are available for specifying the rowset schema.

  • Specify the complete schema in the WITH clause. In specifying the rowset schema, you specify the column names, their data types, and their mapping to the XML document.

Specify the column pattern (using the ColPattern parameter in the SchemaDeclaration). The column pattern specified is used to map a rowset column to the XML node identified by rowpattern and also to determine the type of mapping. If ColPattern is not specified for a column, the rowset column maps to the XML node with same name based on the mapping specified by the flags parameter. However, if ColPattern is specified as part of schema specification in the WITH clause, it overwrites the mapping specified in the flags parameter.

  • Specify the name of an existing table in the WITH clause. You can simply specify an existing table name whose schema can be used by OPENXML to generate the rowset.

  • Do not specify the WITH clause. In this case, OPENXML returns a rowset in the edge table format. This is called an edge table because, in this table format, every edge in the parsed XML document tree maps to a row in the rowset.

Edge tables represent the fine-grained XML document structure (for example, element/attribute names, the document hierarchy, the namespaces, processing instructions, and so on) in a single table (Table 3.5). The edge table format allows you to get additional information that is not exposed through the metaproperties. The additional information provided by edge table allows you to store and query the data type of an element/attribute, the node type (element node, attribute node, or a value node), and information about the XML document structure, and possibly to build your own XML document management system.

Using an edge table, you can write stored procedures that take XML documents as a BLOB input, produce the edge table, and then extract and analyze the document on its finest level (find the document hierarchy, element/attribute names, namespaces, processing instructions, and so on).

The edge table also can serve as a storage format for XML documents when mapping to other relational formats does not make sense and an ntext field is not providing enough structural information.

Whenever you would use an XML parser to examine the XML document, you can use edge table to get the same information.

Mapping Between the Rowset Columns and the XML Nodes

In the OPENXML statement, you can optionally specify the type of mapping (attribute-centric, element-centric) between the rowset columns and the XML nodes identified by the rowpattern. This information is used in transformation between the XML nodes and the rowset columns.

There are two ways to specify the mapping flags and ColPattern (you can specify both). The mapping specified by the flags parameter assumes name correspondence where the XML nodes map to corresponding rowset columns with same name. The ColPattern parameter, an Xpath expression, is specified as part of SchemaDeclaration in the WITH clause. The mapping specified in ColPattern overwrites the mapping specified by the flags parameter. ColPattern can be used to specify the special nature of the mapping (in case of attribute-centric and element-centric mapping) that overwrites or enhances the default mapping indicated by the flags.

ColPattern is specified if the column name in the rowset is different from the element/attribute name to which it is mapped. In this case ColPattern is used to identify the XML element/attribute name to which the rowset column maps.

In mapping a metaproperty attribute to the column, ColPattern is used to identify the metaproperty to which the rowset column maps. Both the flags and ColPattern parameters are optional. If no mapping is specified, attribute-centric mapping (default value of flags parameter) is assumed by default.

If the flags parameter in OPENXML is set to attributes, map to the columns in the rowset based on the name correspondence. Name correspondence means that XML attributes of a given name are stored in a column in the rowset with the same name. If the column name is different from the attribute name to which it maps, ColPattern must be specified. If the XML attribute has a namespace qualifier, the column name in the rowset must have the qualifier as well.

Setting the flags parameter in OPENXML to 2 (XML_ELEMENTS) specifies element-centric mapping. It is similar to attribute-centric mapping except for the following differences. The name correspondence of the mapping (for example, a column mapping to an XML element with the same name) chooses the noncomplex subelements, unless a column-level pattern is specified. In the retrieval case, if the subelement is complex (contains further subelements), the column is set to NULL. Attribute values of the subelements are disregarded.

Multiple subelements with the same name overwrite each other in the order retrieved. Fusion on the parent appends a subelement in case of name equivalence.

Specifying Metaproperties in OPENXML

Metaproperty attributes in an XML document are attributes that describe the properties of an XML item (element, attribute, or any other DOM node). These attributes do not physically exist in the XML document text; however, OPENXML provides metaproperties for all the XML items. These metaproperties allow the extraction of information (such as local positioning and namespace information) from XML nodes, which provide more details than is visible in the textual representation.

You can map metaproperties to the rowset columns in an OPENXML statement using the ColPattern parameter. The columns will contain the values of the metaproperties to which they are mapped. To access the metaproperty attributes, a namespace specific to SQL Server 2000 (urn:schemas- microsoft-com:xml-metaprop) is provided that allows the user to access the metaproperty attributes. If the result of an OPENXML query is returned in an edge table format, the edge table contains one column for each metaproperty attribute (except for the xmltext metaproperty).

Some of the metaproperty attributes are used for processing purposes. For example, the xmltext metaproperty attribute is used for overflow handling. Overflow handling refers to the unconsumed, unprocessed data in the document. One of the columns in the rowset generated by OPENXML can be identified as an overflow column by mapping it to the xmltext metaproperty using the ColPattern parameter. The column then receives the overflow data (the flags parameter determines whether the column contains only the unconsumed data or everything).

Examples

Mapping the OPENXML rowset columns to the metaproperties creates a rowset view of the sample XML document by using OPENXML, showing how the various metaproperty attributes can be mapped to rowset columns in an OPENXML statement using the ColPattern parameter.

In the OPENXML statement, the id column is mapped to the @mp:id metaproperty attribute indicating that the column contains the system-generated unique XML ID of the element, the parent column is mapped to @mp:parentid, indicating that the column contains the XML ID of the parent of the element, and the parentLocalName column is mapped to @mp:parentlocalname, indicating that the column contains the local name of the parent. Then the SELECT statement returns the rowset provided by OPENXML:

DECLARE @idoc int
DECLARE @doc varchar(1000)
-- Sample XML document
SET @doc ='
<root>
  <Customer cid= "C1" name="Janine" city="Issaquah">
      <Order oid="O1" date="1/20/1996" amount="3.5" />
      <Order oid="O2" date="4/30/1997" amount="13.4">Customer was very
satisfied</Order>
   </Customer>
   <Customer cid="C2" name="Ursula" city="Oelde" >
      <Order oid="O3" date="7/14/1999" amount="100" note="Wrap it blue white red">
          <Urgency>Important</Urgency>
      </Order>
      <Order oid="O4" date="1/20/1996" amount="10000"/>
   </Customer>
</root>
'
-- Create an internal representation of the XML document.
EXEC sp_xml_preparedocument @idoc OUTPUT, @doc
-- Execute a SELECT statement using OPENXML rowset provider.
SELECT *
FROM OPENXML (@idoc, '/root/Customer/Order', 9)
      WITH (id int '@mp:id',
            oid char(5),
            date datetime,
            amount real,
            parentIDNo int '@mp:parentid',
            parentLocalName varchar(40) '@mp:parentlocalname')
EXEC sp_xml_removedocument @idoc

This is the result:

id   oid   date                     amount   parentIDNo   parentLocalName
--   ---   -----------------------  -----    ----------   ---------------
6    O1    1996-01-20 00:00:00.000  3.5      2            Customer
10   O2    1997-04-30 00:00:00.000  13.4     2            Customer
19   O3    1999-07-14 00:00:00.000  100.0    15           Customer
25   O4    1996-01-20 00:00:00.000  10000.0  15           Customer

In retrieving the entire XML document, OPENXML creates a one-column rowset view of the sample XML document. This column (Col1) is mapped to the xmltext metaproperty, making it an overflow column. Therefore, the column receives the unconsumed data, which is the entire document in this case, and then the SELECT statement returns the entire rowset:

DECLARE @idoc int
DECLARE @doc varchar(1000)
SET @doc ='
<?xml version="1.0"?>
<root>
  <Customer cid= "C1" name="Janine" city="Issaquah">
      <Order oid="O1" date="1/20/1996" amount="3.5" />
      <Order oid="O2" date="4/30/1997" amount="13.4">Customer was very
             satisfied</Order>
   </Customer>
   <Customer cid="C2" name="Ursula" city="Oelde" >
      <Order oid="O3" date="7/14/1999" amount="100" note="Wrap it blue
             white red">
     <MyTag>Testing to see if all the subelements are returned</MyTag>
          <Urgency>Important</Urgency>
      </Order>
      <Order oid="O4" date="1/20/1996" amount="10000"/>
   </Customer>
</root>
'
-- Create an internal representation of the XML document.
EXEC sp_xml_preparedocument @idoc OUTPUT, @doc
-- Execute a SELECT statement using OPENXML rowset provider.
SELECT *
FROM OPENXML (@idoc, '/')
   WITH (Col1 ntext '@mp:xmltext')
To retrieve the entire document without the XML declaration, the query
can be specified as:
SELECT *
FROM OPENXML (@idoc, '/root')
   WITH (Col1 ntext '@mp:xmltext')
EXEC sp_xml_removedocument @idoc

The query returns the root element with the name root and the data contained by the root element.


Summary

With XML, you can define standard document types that let you share, transmit, and transform business information and facilitate automated business processes. This chapter barely scratches the surface of what you can do—and do quickly—with XML in SQL Server 2000. This kind of productivity enhancement will make SQL Server 2000 a platform of choice for XML delivery.

Here are some additional resources for XML support in SQL Server:

  • XML Developer Center at http://msdn.microsoft.com/xml

  • http://msdn.microsoft.com/sqlserver/

  • http://www.xml.org

  • http://www.w3.org/XML/


Chapter 4. Engine Enhancements in SQL Server 2000

This chapter describes the new storage engine features in SQL Server 2000 and offers tips for using them, along with some insight into how the storage engine works. Developing a basic understanding of the inner workings of the storage engine can help you get the most out of SQL Server.

Different editions of SQL Server 2000 support a wide variety of systems, ranging in size from a tiny mobile system for a Pocket PC all the way up to a high-availability multiple-terabyte transaction processing or decision support system running on clustered Windows 2000 Datacenter servers. All these systems maintain the flexibility, security, and reliability mission-critical business systems demand.

You can deploy SQL Server 2000 applications for projects of varying purposes and sizes as a result of intelligent, automated storage engine operations. A highly sophisticated architecture improves performance, availability, and scalability.

Reliability and concurrency are enhanced with new algorithms for physical file interaction. These algorithms eliminate the need to run database console commands (DBCCs) as part of regular maintenance. However, DBCC is still available, and the new DBCC CHECK commands can be run without inhibiting online processing.

The storage subsystem, which consists of the physical database files and their layout on disk, supports scaling from very small to very large databases. SQL Server can now support up to 64 GB of physical memory (RAM) and up to 32 processors.

Enhanced administration capabilities help the database administrator (DBA) to automate and centralize server management, permitting easy maintenance of remote servers and applications without the necessity of having a DBA visit every site. Server configuration, managed by a sophisticated algorithm, is dynamically responsive to server usage patterns, freeing the DBA to concentrate on database management and optimization tasks.


The Enhancements

The relational database server of SQL Server 2000 has two main parts: the relational engine and the storage engine. The two engines work independently, interacting with each other through native data access components such as OLE DB. The relational engine provides an interface into the storage engine, which is composed of services to interact with the underlying database storage components and features.

The SQL Server 2000 storage engine offers new features that add conceptual simplicity and physical flexibility while reducing the need for meticulous capacity planning and performance tuning. SQL Server 2000 reacts to its own environment and dynamically adapts to changes in database usage accurately and quickly. This breakthrough in technology has elevated the focus of database administration to the facilitation of data as a service. SQL Server 2000 DBAs can focus on designing a system that is responsive to data flow and usage, rather than spending time tuning individual parameters.

The changes in SQL Server 2000 are built on architectural enhancements introduced in SQL Server 7.0 to provide a foundation for ongoing improvement and innovation. A key goal for the storage engine team was to reduce the amount of time and effort spent tuning the server on a regular basis. Because most tuning parameter settings can be based on database usage, the engine now dynamically adjusts to situations in the database environment according to an adaptive algorithm. This automated flexibility has been implemented for tuning parameters that required constant adjustment and experimentation in earlier versions. You can still manually adjust tuning features, but SQL Server 2000 does more of the work for you. Only a small percentage of SQL Server customers should require any adjustment to the tuning parameters; this type of adjustment should only be performed with careful testing and under the supervision of fully qualified database administrators.

Following are the key enhancements made to the SQL Server 2000 storage engine.

Application lock manager

If you need to control concurrent access to application-defined resources, such as forms, new stored procedures now allow you to lock these resources using the SQL Server application lock manager.

Database console commands (DBCCs)

The DBCC CHECK commands can now run during online processing without blocking updates. New enhancements allow verifying consistency of physical pages to detect hardware-induced errors. In the SQL Server 2000 Enterprise Edition, DBCC now runs in parallel across multiple processors.

Database options

All database options can now be modified using ALTER DATABASE. This ability simplifies administration.

Differential backups

Differential backups are quicker in SQL Server 2000, due to an enhancement that tracks database changes at the extent level.

Dynamic tuning

Using dynamic adaptive algorithms, the server automatically adjusts previously static configuration settings. Administrative control is still available to manage systemwide resources, but it is not usually needed. Manually set parameters adapt dynamically within their constrained boundaries.

In-row text

In tables that include a small, frequently used text column, smaller text values can be stored on the same page with the standard data row rather than on a page of text values. In tables where this text data is accessed frequently, this feature can eliminate a large amount of disk I/O.

Index builds in parallel

In the Enterprise Edition, index builds automatically make use of all processors configured for parallel processing, reducing the time it takes to build an index by as much as a factor of 6 on an eight-processor server. Index builds also take advantage of available resources in memory and tempdb.

Index read-ahead

Index reads have been enhanced to increase performance on index scans.

Index reorganization

Improvements made to DBCC SHOWCONTIG provide more detailed information regarding index fragmentation. A new DBCC command, INDEXDEFRAG, reorganizes index pages online without disrupting database service or incurring any risk to database consistency or recovery.

Descending order key columns on indexes

Individual keys columns in an index can be specified in ascending or descending order.

KILL command

This command now reports completion progress. If this command is waiting for another process, such as a rollback, you can view how much of the command has been executed. This command has been enhanced to allow you to stop Microsoft Distributed Transaction Coordinator (MS DTC) transactions, which are not associated with a specific session.

Large memory support

Windows 2000 technology improves the performance of Enterprise Edition systems that use a large amount of memory. Using the AWE extensions of Windows 2000, SQL Server 2000 can support up to 64 GB of physical memory (RAM).

Locking

The lock manager has been enhanced to detect deadlocks across additional resources such as threads and memory. Concurrency improvements reduce deadlocks. This further enhances scalability in SQL Server 2000.

Logical log marks

Transact-SQL commands can create a bookmark in the log to permit restoration of the database to the point in time indicated by the bookmark. This feature also synchronizes restoration of multiple databases used for the same application.

Online index reorganization

Improvements made to DBCC SHOWCONTIG provide more detailed information regarding index fragmentation. A new DBCC command, INDEXDEFRAG, reorganizes index pages online without disrupting database service or incurring any risk to database consistency or recovery.

Optimized I/O read-ahead

SQL Server 2000 issues multiple serial read-ahead reads at once for each file involved in the scan. The query optimizer uses serial, read-ahead I/O when scanning tables and indexes for improved performance.

Passwords on backups

Backup media and individual backup can be password-protected, preventing an unauthorized user from restoring a backup and gaining access to a database.

Recovery models

Using recovery models, you can select the level of logging in the database, allowing greater flexibility of transaction log management. The recovery model can be altered online to complement varying database use throughout the day.

Shared table scans

In the Enterprise Edition, multiple scans of a table can now take advantage of other ongoing scans of that table, reducing physical I/O to the disk.

Shrinking the log

The command to shrink the log runs immediately in more situations. When the log cannot be shrunk immediately, SQL Server provides constructive feedback on what must be done before continuing with or completing the shrink operation.

Snapshot backups

Support for snapshot backups by third-party vendors has been enhanced. Snapshot backups take advantage of storage technologies to back up or restore an entire database in seconds. These backups can now be combined with a conventional transaction log and differential backups to provide complete protection for OLTP databases, especially beneficial for moderate to very large databases in which availability is extremely important.

Space-efficient empty tables and indexes

No disk pages are allocated for empty tables and indexes in SQL Server 2000. SQL Server 7.0 allocated as many as three pages for empty tables and indexes.

Top n sort

This new feature optimizes retrieval of top n values (for example, SELECT TOP 5 [image] FROM tablename).

Xlock

SQL Server 2000 provides this new Transact-SQL locking hint. It can be used to explicitly invoke an exclusive transaction-level page or table lock.


Interacting with Data

In SQL Server 2000, the storage engine has been enhanced to provide even more scalability and performance when interacting with the data. Understanding these enhancements can help you use SQL Server more effectively.

The exchange of data begins with a query originating from a user interface or from an automated task. The data request is passed into the relational engine, which interacts with the storage engine to get the data and pass it back to the user. From the perspective of the user—and even the DBA—the functioning of the storage and relational engines is indistinguishable.

Reading Data More Effectively

Data flows between the server and the user through a series of transactions. The application or user initiates the work, and the database passes it to the query processor for completion and then returns the end results. The query processor does the work by accepting, interpreting, and executing SQL statements. For example, when a user session issues a SELECT statement, the following steps occur.

  1. The relational engine compiles and optimizes the statement into an execution plan (a series of steps required to get the data). The relational engine then runs the execution plan. The execution steps involve accessing tables and indexes through the storage engine.

  2. The relational engine interprets the execution plan, making calls into the storage engine to gather the necessary data.

  3. The relational engine combines all the data returned by the storage engine into the final result set and sends it back to the user.

A couple of improvements have been made to boost performance in this process. In SQL Server 2000, the relational engine relays qualifying query predicates to the storage engine so that they can be applied earlier in the process, resulting in more efficient exchange between the storage and relational engine. This can provide a significant performance gain for qualifying queries.

Top n Enhanced

Another improvement is in the way the storage engine handles selection of the top n records from a result set. In SQL Server 2000, a new top n engine analyzes the best path of operation for statements like this one:

SELECT top 10 * from MyTable order by create_date_time desc

For this example, if the whole table must be searched, the engine analyzes the data and tracks only the top n values in the cache. This is a tremendous performance boost for this type of SELECT statement, because only the values in the top n are sorted, rather than the whole table.

Shared Scans

In the SQL Server 2000 Enterprise Edition, two or more queries can share ongoing table scans, which can improve performance in very large SQL Server 2000 databases. For example, when a query searches a very large table using an unordered scan, the pages flow through the cache to make room for the data flowing in. If another query were started, a second scan of the same table would incur disk I/O to retrieve those pages again. In an environment where there are frequent table scans, this can cause disk thrashing as both queries search the same data pages.

An optimizing process reduces the amount of disk I/O produced by this type of data access pattern. The first unordered scan of a table will read the data from the disk; instead of having to read the disk again, subsequent unordered scans of the same table can build on the information already in memory.

Shared scans are a feature invoked by the storage engine to assist with queries that have no better available execution plan. The intent of this feature is to assist in frequent reads of very large tables. When the query processor determines that the best execution plan includes a table scan, this feature is invoked. However, while it is possible to use query or index tuning to force shared scans, no performance gain is achieved by forcing a table scan where a well-maintained index would do the job as well or better.

Concurrency

To maintain transactional consistency while many users are interacting with the data, the storage engine locks resources to manage dependencies on rows, pages, keys, key ranges, indexes, tables, and databases. By locking resources while they are being altered, the engine prevents more than one user from altering the same data at the same time. SQL Server locks are dynamically applied at various levels of granularity in order to select the least restrictive lock required for the transaction.

In SQL Server 2000, concurrency improvements further reduce deadlocks and avoidable locking of resources. For example, the lock manager has been enhanced to be aware of other resources that might be in contention, such as threads and memory. This new ability can help a database administrator identify a wider variety of design or hardware limitations.

A new Transact-SQL interface into the lock manager has been introduced to support customized locking logic within programming code. Locks necessary for business logic can be initiated by invoking sp_getapplock within a Transact-SQL batch, which permits specification of an application-defined resource to be locked (for example, a lock on an application resource like a form, instead of a lock on a data row), the mode of locking to use, the timeout value, and whether the scope of the lock should be the transaction or the session. After locks have been initiated with the new application lock manager, they participate in the normal lock management of SQL Server, just as if the storage engine had initiated them, so an application-initiated lock will not remain open if the calling transaction is terminated.

The process by which locks are acquired in SQL Server 2000 takes into account whether or not all the data on the page is committed. For example, if you run a SELECT statement against a table whose data has not changed recently, such as a table in the pubs database, the process does not produce any locks because no active transactions have recently updated the table. The storage engine accomplishes this by comparing the log sequence number on the data page to the current active transactions. In databases where most of the data are older than the oldest active transaction, this can reduce locking significantly, enhancing performance.

While locks protect data during transactions, another process, latching, controls access to physical pages. Latches are very lightweight, short-term synchronization objects protecting actions that do not need to be locked for the life of a transaction. When the engine scans a page, it latches the page, reads the row, gives it back to the relational engine, and then unlatches the page again so that another process can reach the same data. Through a process called lazy latching, the storage engine optimizes access to the data pages by releasing latches only when a page is also requested by another ongoing process. If no ongoing process requests the same data page, a single latch remains valid for the entire operation on that page.

For improving concurrency in a system, focus on the design of the database system and the code objects that touch it. SQL Server 2000 is designed to support multiple terabytes of data and virtually unlimited linear scalability. The role of the DBA is to manage the database life cycle, a cycle of design and optimization of all database components from code to data storage on disk, to ensure that the design continues to meet the service level agreement.


Tables and Indexes

Enhancements have also been made to the physical data structures to allow more flexibility of design and maintenance. As a table or index grows, SQL Server allocates new data pages in sets of eight; these are called extents. A row of data cannot cross pages, so it can hold only 8 KB of data, although associated text, ntext, or image columns can be stored on different pages. Tables that have clustered indexes are physically stored in key order on disk. Heaps are tables that do not have clustered indexes and are not sorted. The records are stored in the order in which they were inserted.

SQL Server 2000 supports indexed views, often called materialized views in other database products. When a clustered index is created on a view, the view ceases to be a derived object and becomes a base object stored in the database with the same structure as a table with a clustered index. An indexed view is useful for storing precalculated values or the result of a complex join in cases where the maintenance cost does not outweigh the performance gain. In SQL Server 2000 Enterprise Edition, the query processor automatically uses an indexed view whenever it would optimize a query plan. Indexed views can improve query speed on data that is rarely changed but is frequently part of a complex join or calculation query.

In-Row Text

In-row text allows you to store small text data in the primary page. For example, if you have a table that has a text column, but the text values are frequently small enough to fit on a normal page with the rest of the row, you can set a threshold on the text column. The threshold determines the size below which data is stored on the primary page rather than a separate text page. Much faster performance results if the majority of the data will fit on the page and only a small percentage is actually large enough to justify the creation of a text page.

To determine when to use this new feature, balance the storage density or how many rows are stored on each data page versus the I/O improvement. For example, you have a text column for comments. In the table, you observe that 20 percent of the text values are large but the other 80 percent are less than 100 bytes. This table may seem like a logical candidate for the in-row text solution; however, you should use in-row text only if the data in that column is accessed frequently. If your users access this table frequently, but they do not look at the comments column unless they are doing special research, using in-row text might not be the best answer. The storage density is reduced because fewer rows per page are stored, and because the table contains more pages, table scan response times would be increased. Therefore, the best case for implementing in-row text is when you have a frequently accessed text column that also happens to have many values smaller than 8 K that could be stored in the row.

New Data Types

SQL Server 2000 introduces three new data types: bigint is an 8-byte integer type; sql_variant allows the storage of data values of different data types; table is useful for optimizing performance. Table variables make more efficient use of tempdb, and are faster than temporary tables. Like other variables, they are scoped to the batch in which they are declared. With functionality nearly identical to temporary tables, table variables perform faster than temporary tables or cursors and make better use of server resources. As a rule, always consider the best way to utilize the resources available on your servers when you create code to interact with a database.

Indexes

Access to data is optimized through the use of indexes. Because indexing requirements are based on usage, incorrect indexing is one of the most common causes of slowness in a database. Standard index maintenance should include periodically verifying the current indexing schema and adjusting it to current system usage by dropping or adding indexes as appropriate.

Several new features in SQL Server 2000 make index maintenance more efficient and easier for administration. These enhancements decrease disk I/O, increasing the performance of index scans. This is especially useful where there is a secondary index available for a range scan.

Building indexes

When you build an index, the storage engine samples the rows and calculates the most efficient way to utilize server resources to build the index. Options allow you to control how indexes are built, so you can choose to control how system resources are allocated. You can use these options to balance resources in a process that is important to performance of the system as a whole, in accordance with your knowledge of the particular database system, so the index build will have the lowest possible impact on transaction processing.

Defragmenting indexes

SQL Server 2000 supports online reorganization of indexes, a tremendous advancement from earlier versions. Online index reorganization has minimal impact on transaction throughput and can be stopped and restarted at any time without loss of work. The reorganization is accomplished in small increments and is fully recoverable.

As information is inserted, deleted, and updated in a table, the clustered and nonclustered index pages can eventually become fragmented, decreasing the efficiency of range queries against that data. Therefore, it can be beneficial to defragment your indexes periodically. You can use DBCC SHOWCONTIG, which has been improved in SQL Server 2000, to analyze and report fragmentation. If you determine that an index is fragmented, use DBCC INDEXDEFRAG to reorganize it. It reorders the pages in logical key order, compacting free space and moving rows within the established extents to conform to the fill factor setting. This enhances read performance by densely populating the pages so that fewer of them must be read during a scan of the data. Running DBCC INDEXDEFRAG has far less impact on online performance than rebuilding the index does, provided the index has been regularly maintained and is not completely fragmented.

DBCC INDEXDEFRAG is one of a number of long-running online administrative operations that use small transactions internally. These small transactions maximize concurrency within the server, allow the operation to be stopped without loss of work, and are fully logged to prevent having to redo them in case of a failure.


Logging and Recovery

The transaction log is a stream of records that records changes to the database from the point the database was created until the current point in time. Every logged operation creates a log record. The log records generated by a transaction are written to disk when the transaction commits. In contrast, the data pages modified by the transaction are not immediately written to disk but are retained in the SQL Server buffer cache and written to disk some time later. Delaying writes of the data to disk maximizes the efficiency of multiple accesses to the data pages and avoids disrupting scans. Forcing the log to disk on commit guarantees that no committed work is lost if the server goes down.

Recovery ensures that a database is transactionally consistent prior to bringing it online. If a database is transactionally consistent, all committed work is present and any uncommitted work has been undone. The log always defines the correct view of the database. Simply put, recovery is the process of making the data consistent with the transaction log at a given time.

Recovery is performed automatically when SQL Server starts, when a database is attached, or as the final step in restoring a database from backups. Recovery performed by SQL Server when it starts is called startup recovery. Recovery from backups is normally due to disk failure. This type of recovery is called media recovery.

Restart recovery is automatic and always recovers to the most recent point in time. In the case of recovery from backups, the DBA may choose to recover to an earlier point in time subject to restrictions.

Startup recovery occurs automatically each time an instance of SQL Server is started and consists of rolling back any transactions that were incomplete when the instance was last shut down. In the case of recovery from backups, the DBA may choose to recover to an earlier point in time subject to restrictions. In both cases, recovery operates based on this target time.

Recovery consists of two phases: (1) redo all changes until the target point in time is encountered in the transaction log; (2) undo all work performed by transactions that were active at the point where redo stopped. SQL Server uses checkpoints to speed restart recovery. A checkpoint forces all modified data pages currently in the buffer cache to disk. This creates a starting point for the redo portion of recovery. Because checkpoints can be expensive, SQL Server automatically manages checkpoints to maximize performance while minimizing the time it takes to restart.

In SQL Server 2000, writes that complete successfully must be stored durably on disk. If you use write-caching disk storage, work with your storage vendor to ensure that the cache is fault-tolerant. Fault tolerance means that the cache is immune to power failures or operator actions. If your cache is not fault-tolerant, it should be disabled.

Logical Log Marks

In SQL Server 7.0, it was possible to recover to any specified time. In the case of hardware failure, the restore process was fairly straightforward. However, another threat to a database is the possibility that invalid data may be entered or that valid data may be destroyed by a user's action. In this case, you need to determine when the problem transaction began. In SQL Server 7.0, the only way to do this was to restore logs to a copy of the database until the problem recurred; then you could run your restore to the production image up to a time just prior to the discovered time of error.

In SQL Server 2000, you can mark transactions in the log. Later, if you need to restore, you can refer to the mark that was used at the time of execution rather than using clock time. To do this, use a named BEGIN TRANSACTION statement and the WITH MARK clause. The marks are stored in msdb. Recovery can include or stop right before a transaction that contains the mark. For example, if you have a process that runs in batches and changes many records, you can use this feature to ensure that if the process is run under the wrong circumstances, you can roll the data back to the time the command was executed.

Mark names do not need to be unique. To indicate which transaction you need, specify a datetime value. The syntax for this is

RESTORE LOG WITH [ STOPBEFOREMARK|STOPAFTERMARK ] = @TaggedTransaction AFTER
@datetime

You can also use marks in a distributed transaction, known as distributed marks, to support recovery of multiple related databases to a transactionally consistent state. These related databases might reside on the same or different instances of SQL Server. You can set distributed marks across a set of databases periodically (for example, once every 5 minutes). If the transaction log of one of the databases is damaged, you must recover the set of databases to an earlier time. The distributed mark provides the point in time. Using distributed marks negates the worry of coordinating precise timing of backups for multiple related databases.

Shrinking the Transaction Log

Log shrink operations were not executed immediately in SQL Server 7.0. They were deferred until the transaction log was next backed up or truncated. This confused many SQL Server 7.0 customers. SQL Server 2000 shrinks the log as much as possible and then indicates whether further shrinking will be possible after a log backup. In this case, run the shrink command again after the log backup has completed. The size of the log will be based on your current recovery model and your application design. If you find that you need to shrink the log periodically, look beyond the symptom to the cause. You should investigate what is causing the log to fill up, rather than focus on constant maintenance with the shrink command.

Recovery Models

Recovery models were added to SQL Server 2000 to facilitate data protection planning. They clarify tradeoffs between performance, log space requirements, and protection from media (disk) failure. There are three models: simple recovery, full recovery, and bulk-logged recovery.

The choice of recovery model is based on database usage and availability requirements and helps determine appropriate backup and restore procedures.

Recovery models apply only to media recovery, that is, recovery from backups. Restart recovery recovers all committed work. You can easily change between recovery models. For example, on a very large database, you can use full or bulk-logged recovery, or both. You can use full recovery during the day and bulk-logged recovery at night during a data load process that consists of bulk insert and rebuilding indexes. You can also switch to bulk logging while you run a data load and switch back to full mode, run a transaction log backup, and restore to that point without having to run a full database backup. This feature allows you to do the bulk processing more efficiently; all you need to do is make a transaction log backup afterwards.

To change recovery models, use the following syntax:

ALTER DATABASE SET RECOVERY RecoveryModel
Simple recovery model

The simple recovery model typically requires less log space, but it incurs the greatest potential work loss if data or log files are damaged. Only events needed for basic recovery are logged. Using the simple recovery model, only full database and differential database backups are available. In the event of a failure, all committed work since the last backup must be redone. This model is the simplest to administer, but it is not a good choice for a mission-critical application where loss of committed work cannot be tolerated. This model is similar to the truncate log on checkpoint option in SQL Server 7.0 and earlier versions.

Full recovery model

In the full recovery model, everything is logged. The full recovery model provides complete protection against work loss from a damaged data file. If the transaction log is damaged, work committed since the most recent log backup is lost and must be redone manually. Even when you use the full recovery model, it is important to use fault-tolerant disks for the transaction log to prevent data loss. The full recovery model also allows recovery to any specific time.

Bulk-logged recovery model

The bulk-logged recovery model provides the highest performance for bulk operations. These operations also consume less log space than they do under the full recovery model. For example, the allocation of a new page is logged, but the data inserted onto the page is not. In SQL Server 2000, bulk operations consist of bulk load (BCP and BULK INSERT, including when they run within a DTS package), SELECT INTO, CREATE INDEX, WRITETEXT, and UPDATETEXT.

Compared with the full recovery model, the bulk-logged recovery model minimizes logging for bulk operations. Keep in mind that in the event that recovery becomes necessary, if the log is damaged or if bulk operations have occurred since the most recent log backup, changes made in the database since the last log backup are lost.

This model does not support recovery to a specific time, but it will allow recovery to the end of a transaction log backup containing bulk changes. Transaction log backups made using the bulk-logged recovery model contain the extents modified by bulk operations. This feature improves support for log shipping, because you no longer need to worry that a bulk operation will invalidate your backups. SQL Server maintains a bitmap to track the data extents modified, which optimizes the process by which SQL Server identifies changes.


Improved Backup Functionality

In addition to the introduction of recovery models to simplify data protection in general, SQL Server 2000 has improved manageability: snapshot technology, differential backups, and security have been enhanced. The transaction log backup chain is never broken. In SQL Server 7.0, certain operations, such as adding a file to a database, broke the log chain and required a subsequent full database backup.

Backup operations do not conflict with applications or other administrative actions. For example, backups can occur concurrently with bulk operations such as create index and bulk load.

Log and file backups can occur concurrently. Unattended backup operations, regardless of system activity, are also well supported in SQL Server 2000.

SQL Server supports snapshot backup and restore technologies in conjunction with independent hardware and software vendors. Snapshot backups minimize or eliminate the use of server resources to accomplish the backup. This is especially beneficial for moderate to very large databases in which availability is extremely important. Following are the primary benefits of this technology.

  • A backup can be created in a very short time, usually seconds, with little or no impact on the server.

  • A disk backup can be used to restore a database just as quickly.

  • Another host can create a backup with no impact on the production system.

  • A copy of a production database can be created instantly for reporting or testing.

  • Snapshot backups and restores are accomplished in cooperation with third-party hardware and/or software vendors who use features of SQL Server 2000 designed for this purpose. The backup technology creates an instantaneous copy of the data being backed up, usually by splitting a mirrored set of disks. At restore time, the original is immediately available. The underlying disks are synchronized in the background, resulting in almost instantaneous restores.

  • Differential database backups can be completed in a time that is proportional to the amount of data changed since the last full backup. The less your data has changed, the quicker the backup. SQL Server 2000 uses a bitmap to track data extents modified since the most recent database or file backup to enable them to be located efficiently. In addition, SQL Server 2000 supports file differential backups.

Backups still accumulate changes made to the database since the most recent full backup, functioning the same way in the event of recovery. They are significantly faster, however, because they record only the small amount of information that has changed, especially for very large databases that contain only a small amount of changed data. For added security, you can implement password protection for your backup media and backup sets. This helps prevent unauthorized users from adding to your backups or restoring to your database.


Administrative Improvements

Several administrative features of the storage engine have been enhanced in SQL Server 2000.

Database Verification

The DBCCs provide a variety of administrative capabilities, including the CHECK commands for verifying database consistency. Experience with SQL Server 7.0 and SQL Server 2000 has shown that database inconsistency is caused by hardware problems that may or may not be detected by the database engine or applications during normal operation. This is particularly applicable to data that is accessed infrequently. In response to this need, SQL Server 2000 introduces a checking mode, Physical_Only, designed to detect most hardware-caused problems. It is very fast, approximately disk scan speed, and is not resource intensive.

Due to fundamental architectural improvements in the SQL Server storage engine, which started with SQL Server 7.0, it is not necessary to run database verification as part of normal maintenance. However, Microsoft remains committed to database verification tools as an important part of managing mission critical data. Microsoft recommends that you: (1) run the Physical_Only check occasionally, depending on your confidence in underlying hardware, particularly the disk subsystems, and (2) run a complete database check at critical times, such as a hardware or software upgrade, or whenever a problem is suspected regardless of cause. Microsoft does not recommend running a complete check as part of regular maintenance.

SQL Server 2000 also includes important enhancements to database verification. By default, checking is fully online. Online checking has low impact on the transaction workload. This impact will vary depending on the system load, hardware configuration, and speed of tempdb. Microsoft has measured this impact at 15 to 20 percent with a medium OLTP workload (50 percent CPU). The TABLOCK option is provided to force the check to take shared table locks, which enables it to run faster but will prevent updates.

Checking is done in parallel on symmetric multiprocessing (SMP) computers, limited by the maximum degree of parallelism you have set for the instance of SQL Server. SQL Server 2000 check commands continue to support the repair functionality introduced in SQL Server 7.0. Offline repair can provide an alternative to a restore from backups in some situations.

Database State Control

SQL Server 2000 includes enhancements to the ALTER DATABASE statement that allow more control of database states through Transact-SQL. All database options can now be modified with greater control through the ALTER DATABASE command; sp_dboption and databaseproperty() will no longer be updated in future releases. The Transact-SQL commands sp_helpdb and DatabasePropertyEx() provide information about the state of your database.

SQL Server also sets the following states in reaction to conditions within the database: restoring, recovering, and suspect. The database options can be set by using the SET clause of the ALTER DATABASE statement, the sp_dboption system stored procedure, or, in some cases, SQL Server Enterprise Manager.

When the database state is changed, the session making changes to the database state remains connected, while sessions inconsistent with the new state can be terminated and their transactions rolled back. Session termination options include the following:

  • Terminate immediately.

  • Terminate after a specified time.

  • Allow the ongoing processes to complete normally.

  • Check for activity and disregard the state change if active user sessions are found.

Here are two examples of the syntax:

alter database accting set read_only with rollback immediate
alter database accting set single_user with rollback after 60 seconds

System Process IDs and Units of Work

One additional administrative enhancement that helps when you need to stop a process is the KILL command. The KILL command has been enhanced with status feedback. So if you want to learn the status of an outstanding KILL command, run the following:

KILL SPID WITH STATUSONLY

If you try to stop a system process ID (SPID) that is being stopped by another KILL command, the system returns the same status information.

In SQL Server 2000, MS DTC transactions can exist without an associated connection or SPID. Therefore, a connection can be used for other processes while waiting for a transaction or unit of work to complete. When the MS DTC transaction manager sends a message that it has completed the task, you can either commit or roll back the transaction. This is referred to as a unit of work (UOW), which is the transaction identifier used by MS DTC for the transaction. A UOW does not have a SPID.


Dynamic Tuning

In SQL Server 2000, usage-based performance tuning is managed dynamically, without required or recommended manual adjustments. The static parameters have been eliminated, but administrative control has been retained for certain resources (for example, setting an upper limit on the amount of memory SQL Server can use). This method is far more accurate and responsive than a manually calculated system based on averages and estimates. It allows you to concentrate on the design aspects of database management. Traditional database systems require a great deal of manual management and tuning. For example, to tune the system in response to usage, the DBA would be required to monitor the system, recording a vast amount of statistics over time, in order to select a static setting that seems to provide the optimal advantage for the system. Then the DBA would reevaluate the system to judge what effect the new setting has, and the tuning process would begin again. SQL Server 2000 introduces into the storage engine a dynamic algorithm that actively monitors usage of the server and adjusts the settings internally.


Data Storage Components

SQL Server 2000 balances processing across all available CPUs in coordination with the Windows 2000 operating system. If you are running a dedicated instance of SQL Server, and no other applications produce a load on the same resources, leave the processor-related settings at their default to make full use of all the processors. SQL Server can take advantage of parallel processing across multiple processors for queries, index builds, DBCCs and other operations.

The SQL Server 2000 Standard Edition can support up to four processors and 2 GB of physical memory (RAM). The Enterprise Edition can scale upward to new levels to support up to 32 processors and 64 GB of physical memory (RAM).

Memory Pool

The main source of memory for an instance of SQL Server is called its memory pool. Almost all data structures that use memory in an instance of SQL Server are allocated from the memory pool. Examples of objects allocated from the memory pool include the buffer cache, where recently read data pages are stored, and the procedure cache, which holds recent execution plans.

The assignments within the memory pool are highly dynamic. To optimize performance, SQL Server constantly adjusts the amounts of the memory pool assigned to the various areas. For example, when the number of stored execution plans is lower, the memory pool is adjusted to make optimal use of the resources by making more memory available for the data cache.

SQL Server 2000 is designed to use memory to minimize disk I/O as much as possible. To do so, SQL Server uses the buffer cache to hold recently referenced data in physical memory (RAM) where it can be reused. One potential way to reduce disk I/O and speed up a database system would be to add to the physical memory (RAM) available to SQL Server.

Normally, memory settings do not require any adjustment. However, they can be controlled in certain situations. For example, memory requires special attention where you are running multiple instances of SQL Server on the same server, especially if you use failover clustering. You also need to monitor memory usage if you are running applications in addition to SQL Server on the same server.

SQL Server 2000 takes advantage of new capabilities in Windows 2000 to address physical memory (RAM) beyond 3 GB. The SQL Server 2000 Enterprise Edition can use as much memory as Windows 2000 Advanced Server and Windows 2000 Datacenter Server allow.

Files, Filegroups, and Disks

SQL Server stores data and the log in disk files. In a basic installation and as a default, data and log files are created in the default location specified in the server configuration. However, to maximize performance and manageability, you can apply a few basic principles:

  • Spread data over as many disks, channels, and controllers as possible.

  • In general, the more disks (spindles, regardless of their individual size) and the faster the access to them (controllers and channels), the faster the storage engine can read and write data. The larger the system usage becomes, the more important it is to separate the data files from log files by storing them on different sets of physical drives. Also, because the use of tempdb has changed, tempdb should be stored on a separate large set of disks, for example, with the data files or on a set of disks.

  • Use filegroups to make an enterprise database more manageable.

  • Every database begins with one default filegroup. Because SQL Server 2000 can work effectively without additional filegroups, many systems will not need to add user-defined filegroups. However, as a system grows, the use of additional filegroups can provide more manageability when implemented and maintained by a qualified DBA.

In SQL Server 2000, if you set a particular filegroup within a database to read only, the data on that filegroup cannot be altered, but catalog information such as permissions can still be managed. (Note: In SQL Server 2000, the number of asynchronous I/Os is now managed dynamically inside the database engine and is not influenced by the number of files or filegroups used, as was the case in SQL Server 7.0.) When implementing or optimizing a database design, the database administrator (database system engineer) needs to consider the configuration of the database storage components, particularly the layout of physical and logical disks and the arrangement of the database files across disks.


Summary

For DBAs, increased flexibility and control over performance provides the freedom to focus their database technology skills and experience on managing the database code, design, and storage components as a unified approach to database system management. The SQL Server 2000 database engine provides general extensibility and flexibility for a wide variety of database implementations.


Chapter 5. DTS Enhancements

Most organizations have multiple formats and locations in which data is stored. To support decision making, improve system performance, or upgrade existing systems, data often must be moved from one data storage location to another.

Data Transformation Services (DTS) is a set of tools you can use to import, export, and transform heterogeneous data between one or more data sources, such as Microsoft SQL Server, Microsoft Excel, or Microsoft Access. Connectivity is provided through OLE DB, an open standard for data access. ODBC (open database connectivity) data sources are supported through the OLE DB provider for ODBC.

A DTS solution is created as one or more packages. Each package may contain an organized set of tasks that define work to be performed, transformations on data and objects, workflow constraints that define task execution, and connections to data sources and destinations. DTS packages also provide services, such as logging package execution details, controlling transactions, and handling global variables.

The following tools are available for creating and executing DTS packages:

  • Import/Export Wizard is for building relatively simple DTS packages, and it supports data migration and simple transformations.

  • DTS Designer graphically implements the DTS object model, allowing you to create DTS packages with a wide range of functionality.

  • DTSRun is a command prompt utility used to execute existing DTS packages.

  • DTSRunUI is a graphical interface to DTSRun, which also allows the passing of global variables and the generation of command lines.

  • SQLAgent is not a DTS application; however, it is used by DTS to schedule package execution.

Using the DTS object model, you also can create and run packages programmatically, build custom tasks, and build custom transformations.

DTS provides a set of tools that lets you extract, transform, and consolidate data from disparate sources into single or multiple destinations. By using DTS tools, you can create custom data movement solutions tailored to the specialized needs of your organization, as shown in the following scenarios:

  • You have deployed a database application on an older version of SQL Server or another platform, such as Microsoft Access. A new version of your application requires SQL Server 2000 and requires you to change your database schema and convert some data types. To copy and transform your data, you can build a DTS solution that copies database objects from the original data source into an SQL Server 2000 database, while at the same time remapping columns and changing data types. You can run this solution using DTS tools, or you can embed the solution within your application.

  • You must consolidate several key Microsoft Excel spreadsheets into an SQL Server database. Several departments create the spreadsheets at the end of the month, but there is no set schedule for completion of all the spreadsheets. To consolidate the spreadsheet data, you can build a DTS solution that runs when a message is sent to a message queue. The message triggers DTS to extract data from the spreadsheet, perform any defined transformations, and load the data into a SQL Server database.

  • Your data warehouse contains historical data about your business operations, and you use Microsoft SQL Server 2000 Analysis Services to summarize the data. Your data warehouse needs to be updated nightly from your online transaction processing (OLTP) database. Your OLTP system is in use 24 hours a day, and performance is critical.

You can build a DTS solution that uses the file transfer protocol (FTP) to move data files onto a local drive, load the data into a fact table, and aggregate the data using Analysis Services. You can schedule the DTS solution to run every night, and you can use the new DTS logging options to track how long this process takes, allowing you to analyze performance over time.


New Features in DTS

Microsoft SQL Server 2000 introduces the following DTS enhancements and new features.

New Custom Tasks

New DTS custom tasks, available through DTS Designer or the DTS object model, allow you to create DTS packages that perform tasks or set variables based on the properties of the runtime environment. Use these tasks to

  • import data from and send data and completed packages to Internet and file transfer protocol (FTP) sites.

  • run packages asynchronously.

  • build packages that send messages to each other.

  • build packages that execute other packages.

  • join multiple package executions as part of a transaction.

Enhanced Logging Facilities

DTS package logs save information for each package execution, allowing you to maintain a complete execution history. You can also view execution information for individual processes within a task. You can generate exception files for transformation tasks. When you log to exception files, you can save source and destination error rows to a file through the DTS OLE DB text file provider and reprocess the error rows.

Saving DTS Packages to Visual Basic Files

DTS packages now can be saved to a Microsoft Visual Basic (VB 6.0) file. This allows a package created by the DTS Import/Export Wizard or DTS Designer to be incorporated into Visual Basic programs or to be used as prototypes by Visual Basic developers who need to reference the components of the DTS object model. Actually, the utility existed in SQL Server 7.0, wherein you could have used the ScriptPkg utility to save a DTS package as a VB file.

Using the Multiphase Data Pump

A new multiphase data pump allows advanced users to customize the operation of the data pump at various stages of its operation. You can now use global variables as input and output parameters for queries.

Using Parameterized Queries

You can now use parameterized source queries in a DTS transformation task and an Execute SQL task. In addition, DTS includes an option for saving the results of a parameterized query to a global variable, allowing you to perform functions such as saving disconnected Microsoft ActiveX Data Objects (ADO) recordsets in DTS.

Using Global Variables to Pass Information Between DTS Packages

You now can use the Execute Package task to dynamically assign the values of global variables from a parent package to a child package. Use global variables to pass information from one package to another when each package performs different work items. For example, use one package to download data nightly, summarize the data, assign summary data values to global variables, and pass the values to another package that further processes the data.


Using DTS Designer

DTS Designer graphically implements the DTS object model, permitting the graphic creation of DTS packages. You can use DTS Designer to

  • create a simple package containing one or more steps.

  • create a package that includes complex workflows that include multiple steps using conditional logic, event-driven code, or multiple connections to data sources.

  • edit an existing package.

The DTS Designer interface consists of a work area for building packages, toolbars containing package elements that you can drag onto the design sheet, and menus containing workflows and package management commands.

By dragging connections and tasks onto the design sheet and specifying the order of execution with workflows, you can easily build powerful DTS packages using DTS Designer. The following sections define tasks, workflows, connections, and transformations, and illustrate the ease of using DTS Designer to implement a DTS solution.

Tasks: Defining Steps in a Package

A DTS package usually includes one or more tasks. Each task defines a work item that may be performed during package execution (Table 5.1).

Table 5.1. DTS package tasks
Icon | Task | Description
Transforming data
Transform Data | Move data between a source and destination and to optionally apply column-level transformations to the data.
Data Driven Query | Perform flexible, Transact-SQL–based operations on data, including stored procedures and INSERT, UPDATE, or DELETE statements.
Parallel Data Pump[1] | Available programmatically only, the Parallel Data Pump task performs the same functions as the Transform Data and Data Driven Query tasks, but it supports chaptered rowsets as defined by OLE DB 2.5 and later.
Copying and managing data
Bulk Insert | Quickly load large amounts of data into an SQL Server table or view.
Execute SQL | Run SQL statements during package execution. The Execute SQL task also can save data that is the result of a query.
Copy SQL Server Objects | Copy SQL Server objects from one installation or instance of SQL Server to another. You can copy objects such as data and tables, as well as the definitions of objects such as views and stored procedures.
Transfer Database[1] | Move or copy an SQL Server database from an instance of SQL Server 7.0 or SQL Server 2000 to an instance of SQL Server 2000.
Transfer Error Messages[1] | Copy user-specified error messages, created by the sp_addmessage system stored procedure, from an instance of SQL Server 7.0 or SQL Server 2000 to an instance of SQL Server 2000.
Transfer Logins[1] | Copy logins from an instance of SQL Server 7.0 or SQL Server 2000 to an instance of SQL Server 2000.
Transfer Jobs[1] | Copy jobs from an instance of SQL Server 7.0 or SQL Server 2000 to an instance of SQL Server 2000.
Transfer Master Stored Procedures[1] | Copy stored procedures from a master database on an instance of SQL Server 7.0 or SQL Server 2000 to the master database on an instance of SQL Server 2000.
Running tasks as jobs from within a package
ActiveX Script | Write code to perform functions that are not available in the other DTS tasks.
Dynamic Properties[1] | Retrieve values from sources outside a DTS package at package runtime and assign those values to selected package properties.
Execute Package[1] | Run other DTS packages from within a package.
Execute Process | Run an executable program or batch file.
File Transfer Protocol (FTP) task[1] | Download data files from a remote server or an Internet location.
Message Queue[1] | Send and receive messages from Microsoft Message Queues.
Send Mail | Send an e-mail message.
Analysis Services Processing[2] | Perform processing of one or more objects defined in SQL Server 2000 Analysis Services.
Data Mining[1],[2] | Create a prediction query and an output table from a data mining model object defined in SQL Server 2000 Analysis Services.

[1] New in SQL Server 2000.

[2] Available only when SQL Server 2000 Analysis Services is installed.

You also can create custom tasks programmatically and then integrate them into DTS Designer using the Register Custom Task command.

To illustrate the use of tasks, a simple DTS Package might have two tasks:

The ActiveX Script task can host any ActiveX Scripting engine including Microsoft Visual Basic Scripting Edition (VBScript), Microsoft JScript, or ActiveState ActivePerl, which can be downloaded from http://www.activestate.com. The Send Mail task may send a message indicating that the package has run. Note that there is no order to these tasks yet. When the package executes, the ActiveX Script task and the Send Mail task run concurrently.

Workflows: Setting Task Precedence

When you define a group of tasks, there is usually an order in which the tasks should be performed. When tasks have an order, each task becomes a step of a process. In DTS Designer, you manipulate tasks on the design sheet and use precedence constraints to control the sequence in which the tasks execute.

Precedence constraints sequentially link tasks in a package. Table 5.2 shows the types of precedence constraints you can use in DTS.

Figure 5.1 illustrates the ActiveX Script task and the Send Mail task with an On Completion precedence constraint. When the Active X Script task completes with either success or failure, the Send Mail task runs.You can configure separate Send Mail tasks, one for an On Success constraint and one for an On Failure constraint (Figure 5.2). The two Send Mail tasks can send different messages based on the success or failure of the ActiveX script. You also can issue multiple precedence constraints on a task. For example, the Send Mail task "Admin Notification" could have both an On Success constraint from Script 1 and an On Failure constraint from Script 2 (Figure 5.3). In these situations, DTS assumes a logical "AND" relationship. Therefore, Script 1 must successfully execute and Script 2 must fail for the Admin Notification message to be sent.

Figure 5.1. On Completion procedure constraint

Figure 5.2. On Success precedence constraint plus On Failure precedence constraint

Figure 5.3. Multiple precedence constraints

Table 5.2. DTS precedence constraints
Precedence constraint | Description
On Completion (blue arrow) | If you want Task 2 to wait until Task 1 completes, regardless of the outcome, link Task 1 to Task 2 with an On Completion precedence constraint.
On Success (green arrow) | If you want Task 2 to wait until Task 1 has successfully completed, link Task 1 to Task 2 with an On Success precedence constraint.
On Failure (red arrow) | If you want Task 2 to begin execution only if Task 1 fails to execute successfully, link Task 1 to Task 2 with an On Failure precedence constraint.

Connections: Accessing and Moving Data

To successfully execute DTS tasks that copy and transform data, a DTS package must establish valid connections to its source and destination data and to any additional data sources, such as lookup tables.

When creating a package, you configure connections by selecting a connection type from a list of available OLE DB providers and ODBC drivers. The following types of connections are available.

  • Microsoft Data Access Components (MDAC) drivers

    Microsoft OLE DB Provider for SQL Server

    Microsoft Data Link

    Microsoft ODBC Driver for Oracle

  • Microsoft Jet drivers

    dBase 5

    Microsoft Access

    HTML File (Source)

    Microsoft Excel 97–2000

    Paradox 5.X

  • Other drivers

    Text File (Source)

    Text File (Destination)

    Other Connection

DTS allows you to use any OLE DB connection. The icons on the Connections toolbar provide easy access to common connections.


Data Pump: Transforming Data

The DTS data pump is a DTS object that drives the import, export, and transformation of data. The data pump is used during the execution of the Transform Data, Data Driven Query, and Parallel Data Pump tasks. These tasks work by creating rowsets on the source and destination connections, then creating an instance of the data pump to move rows between the source and destination. Transformations occur on each row as the row is copied (Table 5.3). In the transformation tasks, any changes to be made to data are also defined.

You can also create your own custom transformations programmatically. The quickest way to build custom transformations is to use the Active Template Library (ATL) custom transformation template, which is included in the SQL Server 2000 DTS sample programs.

Table 5.3. Built-in transformations provided by DTS
Transformation | Use
Copy Column | Copy data directly from source to destination columns without any transformations applied to the data.
ActiveX Script | Build custom transformations. Note that since the transformation occurs on a row-by-row basis, an ActiveX script can affect the execution speed of a DTS package.
DateTime String | Convert a date or time in a source column to a different format in the destination column.
Lowercase String | Convert a source column to lowercase characters and, if necessary, to the destination data type.
Uppercase String | Convert a source column to all uppercase characters and, if necessary, to the destination data type.
Middle of String | Extract a substring from the source column, transform it, and copy the result to the destination column.
Trim String | Remove leading, trailing, and embedded white space from a string in the source column and copy the result to the destination column.
Read File | Open the contents of a file whose name is specified in a source column, and copy the contents into a destination column.
Write File | Copy the contents of a source column (data column) to a file whose path is specified by a second source column (file name column).

Data Pump Error Logging

A new method of logging transformation errors is available in SQL Server 2000. You can define three exception log files for use during package execution: an error text file, a source error rows file, and a destination error rows file. General error information is written to the error text file. If a transformation fails, then the source row is in error, and that row is written to the source error rows file. If an insert fails, then the destination row is in error, and that row is written to the destination error rows file. The exception log files are defined in the tasks that transform data. Each transformation task has its own log files.

Data Pump Phases

By default, the data pump has one phase: row transformation. That phase is configured when mapping column-level transformations in the Transform Data task, Data Driven Query task, and Parallel Data Pump task, without selecting a phase. Multiple data pump phases are new in SQL Server 2000. By selecting the multiphase data pump option in SQL Server Enterprise Manager, the data pump can be accessed at several points during its operation and add functionality.

After the data pump processes the last row of data, the task is finished and the data pump operation terminates. Advanced users who want to add functionality to a package so that it supports any data pump phase can do so by either writing an ActiveX script phase function for each data pump phase to be customized or by creating a COM object in Microsoft Visual C++ to customize selected data pump phases. If you use ActiveX script functions to customize data pump phases, no additional code outside the package is required. Creating a COM object is a program external to the package, and the program is called for each selected phase of the transformation. Unlike the ActiveX script method of accessing data pump phases, which uses a different function and entry point for each selected phase, this method provides a single entry point that is called by multiple data pump phases while the data pump task executes.


Options for Saving DTS Packages

The following options are available for saving DTS packages:

  • Microsoft SQL Server: Save your DTS package to Microsoft SQL Server if you want to store packages on any instance of SQL Server on your network, keep a convenient inventory of the packages, and add and delete package versions during the package development process.

  • SQL Server 2000 Meta Data Services: Save your DTS package to meta data services if you plan to track package version, meta data, and data lineage information.

  • Structured storage file: Save your DTS package to a structured storage file if you want to copy, move, and send a package across the network without having to store the package in a Microsoft SQL Server database.

  • Microsoft Visual Basic: Save your DTS package that has been created by DTS Designer or the DTS Import/Export Wizard to a Microsoft Visual Basic file if you want to incorporate it into Visual Basic programs or use it as a prototype for DTS application development.


DTS as an Application Development Platform

The DTS Designer provides a wide variety of solutions to data movement tasks. DTS extends the number of solutions available by providing programmatic access to the DTS object model. Using Microsoft Visual Basic, Microsoft Visual C++, or any other application development system that supports COM, you can develop a custom DTS solution for your environment using functionality unsupported in the graphical tools.

DTS offers support for the developer in three ways:

  1. Building packages: You can develop extremely complex packages and access the full range of functionality in the object model without the using the DTS Designer or DTS Import/Export Wizard.

  2. Extending packages: You can add new functionality through the construction of tasks and transforms customized for your business and reusable within DTS.

  3. Executing packages: Execution of DTS packages does not have to be from any of the tools provided. It is possible to execute DTS packages programmatically and display progress through COM events, allowing the construction of embedded or custom DTS execution environments.


Summary

In this chapter we covered the new feature sets of DTS and what can be achieved with them. (A sample DTS package is shown in Chapter 27 that demonstrates refreshing one database from the other and backing up the databases.) For additional information, see these resources:

  • Microsoft SQL Server Web site at http://www.microsoft.com/sql.

  • Microsoft SQL Server Developer Center at http://msdn.microsoft.com/sqlserver.

  • microsoft.public.sqlserver.dts newsgroup at news://news.microsoft.com.


Chapter 6. Profiler and Index-Tuning Wizard Enhancements

SQL Profiler is a tool that captures Microsoft SQL Server 2000 events from a server. The events are saved in a trace file that can later be analyzed or used to replay a specific series of steps when trying to diagnose a problem. SQL Profiler is used for activities such as the following:

  • Stepping through problem queries to find the cause of the problem.

  • Finding and diagnosing slow-running queries.

  • Capturing the series of SQL statements that lead to a problem. The saved trace can then be used to replicate the problem on a test server where the problem can be diagnosed.

  • Monitoring the performance of SQL Server to tune workloads.

SQL Profiler also supports auditing the actions performed on instances of SQL Server. Audits record security-related actions for later review by a security administrator.

SQL Profiler now supports size-based and time-based traces and includes new events for Data File Auto Grow, Data File Auto Shrink, Log File Auto Grow, Log File Auto Shrink, Show Plan All, Show Plan Statistics, and Show Plan Text. SQL Profiler has been enhanced to provide auditing of SQL Server activities, up to the auditing levels required by the C2 level of security defined by the United States government.


SQL Profiler Terminology

To use SQL Profiler, you need to understand the terminology that describes the way the tool functions. For example, you create a template that defines the data you want to collect. You collect this data by running a trace on the events defined in the template. While the trace is running, the event classes and data columns that describe the event data are displayed in SQL Profiler.

template A template defines the criteria for each event you want to monitor with SQL Profiler. For example, you can create a template specifying which events, data columns, and filters to use. Then you can save the template and launch a trace with the current template settings. The trace data captured is based on the options specified in the template. A template is now executed and must be saved to a file with the .tdf extension.

trace A trace captures data based on the selected events, data columns, and filters. For example, to create a template to monitor exception errors, you select to trace the Exception event class and the Error, State, and Severity data columns, which need to be collected for the trace results to provide meaningful data. After you save the template, you can then run it as a trace and collect data on any Exception events that occur in the server. The trace data can be saved and then replayed at a later date or used immediately for analysis.

filter When you create a trace or template, you can define criteria to filter the data collected by the event. If traces are becoming too large, you can filter them based on the information you want so that only a subset of the event data is collected. If a filter is not set, all events of the selected event classes are returned in the trace output. For example, you can limit the Microsoft Windows 2000 user names in the trace to specific users, reducing the output data to only those users in which you are interested.

event category An event category defines the way events are grouped. For example, all lock events classes are grouped within the Locks event category. However, event categories exist only within SQL Profiler. This term does not reflect the way engine events are grouped.

event An event is an action generated within the Microsoft SQL Server engine. For example:

  • Login connections, failures, and disconnections

  • Transact-SQL SELECT, INSERT, UPDATE, and DELETE statements

  • Remote procedure call (RPC) batch status

  • Start or end of a stored procedure

  • Start or end of statements within stored procedures

  • Start or end of an SQL batch

  • Error written to the SQL Server error log

  • Lock acquired or released on a database object

  • Opened cursor

  • Security permissions checks

All the data that is generated as a result of an event is displayed in the trace in a single row. This row contains columns of data called event classes that describe the event in detail.

event class An event class is a column that describes the event that was produced by the server. The event class determines the type of data collected, and not all data columns are applicable to all event classes. Examples of event classes include the following:

  • SQL:BatchCompleted, which indicates the completion of an SQL batch

  • Name of the computer on which the client is running

  • ID of the object affected by the event, such as a table name

  • SQL Server name of the user issuing the statement

  • Text of the Transact-SQL statement or stored procedure being executed

  • Time the event started and ended

data column The data columns describe the data collected for each of the event classes captured in the trace. Because the event class determines the type of data collected, not all data columns are applicable to all event classes. For example, the Binary Data column, when captured for the Lock:Acquired event class, contains the value of the locked page ID or row but has no value for the Integer Data event class. Default data columns are populated automatically for all event classes.


Starting Profiler Trace

To define a new SQL Server 2000 Profiler trace, open Profiler from the SQL Server program group and click the New Trace button, the leftmost button in the toolbar—or you can choose File, New, Trace. The resulting Connect to SQL Server dialog box requests your server and security information for logging in to SQL Server. After you connect, you'll see a tabbed dialog box in which you can define general options, the events you want to capture, the data columns you want to receive with the traced events, and any pertinent filters. Many of the options in SQL Server 2000 Profiler are new, including the file-rollover option; server-side processing of trace data, and certain events, data columns, and filtering capabilities. Let's explore the option tabs one by one.

General

On the General tab (Figure 6.1), you can name the trace and change the target server for tracing. You can also choose a trace template that comes with a predefined set of events, data columns, and filters and that lets you reuse a previously saved trace definition. Some trace templates come with SQL Server, but you can also create your own templates. One template that comes with SQL Server is SQLProfilerSP_Counts, which lets you trap invocations of stored procedures.

Figure 6.1. General tab

On the General tab, you can specify a file to which SQL Server saves the trace output. Saving a trace to a file is much more efficient with SQL Server 2000 Profiler than with the Profiler in SQL Server 7.0 because the 2000 trace file uses a self-describing format. The use of a self-describing file format means that for each event, SQL Server 2000 saves only the relevant data columns, not white spaces, and some events simply point to previous events. For example, completed events such as SQL:BatchCompleted and SQL:StmtCompleted can point to their corresponding starting events—SQL:BatchStarting and SQL:StmtStarting—to save storage space. An important advantage of saving the trace output to a file is that when you reopen the file later for examination or for replay, it loads asynchronously—that is, you can start examining the file while it's still loading.

When you specify that SQL Server should save the trace output to a file, you must also set a maximum file size in the range of 1 MB to 1000 MB. In addition, the General tab lets you enable file rollover and specify that the server will process the trace data. If you specify a maximum file size but don't enable the file-rollover option, SQL Server stops writing events to the file after the file reaches the size you specified. Note that because newer events don't overwrite previous events, you get only the first events that fit the specified file size. However, SQL Server continues to write events to the screen. If you enable the file-rollover option, as soon as the trace file reaches the specified maximum size, SQL Server creates a new trace file.

The first trace file's name is the name you specified in the Save to file text box. SQL Server uses three elements to construct the next file's name: the file name you specified, an underscore, and a number, starting with 1 and incremented by 1 with every new file. For example, suppose you specify C:\traces\mytrace.trc as the trace filename and you leave the default maximum file size of 5 MB. If you trace 17.5 MB of trace data, you'll have four files: mytrace.trc, my-trace_1.trc, mytrace_2.trc, and mytrace_3.trc.

The file-rollover option gives you some important capabilities. You can start examining files while Profiler continues to send output to newer files; you don't have to wait until the trace stops before you start analyzing the trace output. Also, if your trace generates large amounts of data and your disk is nearly full, you can move the full inactive files to another disk or even to another server.

By enabling the option "Server processes SQL Server trace data," you specify that the server, not the client, should handle the trace data. This means that a server-side trace file writer, which in SQL Server 2000 is based on the transaction log writer, is responsible for writing the trace data to the file. The default in Profiler is for the client to process the trace data. When you enable the option "Server processes SQL Server trace data," you're guaranteed to capture all events you requested. But this option can degrade the server's performance under stress conditions, when the server is already under a heavy load. However, if you don't enable this option—letting the client handle the trace data instead, as SQL Server 7.0 Profiler does—SQL Server might not capture all events under stress conditions.

On the General tab, you can send the trace output to a table and specify the maximum number of rows (in thousands) that Profiler can write to that table. Saving the trace output to a table lets you analyze the trace data by using T-SQL queries or even a Microsoft Excel pivot table or Analysis Services cube to manipulate the trace data dynamically. Note that even if you send the trace output to a file instead of a table, you can use the function fn_trace_gettable() to return the trace data in table format. The fn_trace_gettable() function accepts the trace filename and optionally a maximum number of files to process as parameters; it returns a rowset containing the trace data. For example, suppose Profiler generated the following three files for a certain trace: C:\traces\mytrace.trc, C:\traces\mytrace_1.trc, and C:\traces\mytrace_2. trc. To copy the content of only the first file to a table, you would use fn_trace_gettable(), specifying 1 in the second parameter:

SELECT * INTO #Mytrace
FROM ::fn_trace_gettable('C:\traces\mytrace.trc', 1)

The last option available on the General tab is "Enable trace stop time," which lets you specify a date and time for the trace to stop capturing events. Note that if you use SQL Server 2000 Profiler to trace a SQL Server 7.0 machine, the options "Server processes SQL Server trace data" and "Enable trace stop time" are unavailable.

Events

On the Events tab (Figure 6.2), you can choose which events you want to include in your trace. SQL Server 2000 Profiler introduced many new events that aren't available with SQL Server 7.0 Profiler and also changed the categorization of some events. Although the list of available events on the Events tab is the same whether you define a trace against SQL Server 2000 or SQL Server 7.0, the events you can capture depend on what the server-side components generate.

Figure 6.2. Events tab

Among the new events in Profiler are Data File Auto Grow, Data File Auto Shrink, Log File Auto Grow, and Log File Auto Shrink in the new Database category. In the Performance category, which is also new, are the Execution Plan event and the Show Plan Text and Show Plan All events. Also in the Performance category are four Degree of Parallelism events, one for each Data Manipulation Language (DML) statement (SELECT, INSERT, UPDATE, DELETE). Note that the four Degree of Parallelism events have different meanings only if you're tracing an SQL Server 7.0 server. If you're tracing an SQL Server 2000 server, any of the four Degree of Parallelism events is sufficient to capture all types of DML statements, so you can include just one of the four. The statement type appears in the EventSubClass data column.

The Security Audit category is also new. It includes 21 database audit events that let you track, among other things, failed or successful login attempts; the addition and removal of logins, database users, and database roles; the use of statement and object permissions; backup and restore operations; and the issuing of DBCC commands. In addition, this category gives you 10 user-configurable events, whereas SQL Server 7.0 Profiler gives only 5.

Starting and Completed events, which you can find in the Stored Procedures and T-SQL categories, are two of the most commonly traced events. In general, Starting events are important when you want to track the existence and chronological order of certain activities. For example, if you want to analyze the types of queries issued against your server during various periods of the day, you need to capture the text of the queries and the time they were issued. For this purpose, capturing a Starting event is sufficient.

Completed events are important when you want to get performance measures of activities such as CPU time and logical disk reads, because performance measures are known only after activities have finished. For example, you would use Completed events to locate slow-running queries. In some situations, you'd want to capture both Starting and Completed events. For example, suppose you're troubleshooting a certain client application that experiences a lot of timeouts, such as "Lock request time out period exceeded" errors. You need to pinpoint the queries that start but never finish. If you capture only Completed events, you won't get the queries that incurred timeouts because those queries never completed. However, you could look for Starting events that don't have corresponding Completed events to find the queries that never finished.

You can divide Starting and Completed events by their level of detail; we call these events less granular (not so detailed) and more granular (very detailed) events. Less granular events include RPC:Starting, RPC:Completed, SP:Starting, SP:Completed, SQL:BatchStarting, and SQL:BatchCompleted. SQL Server generates less granular events when remote procedure calls, stored procedures, and batches start and finish. SQL Server generates more granular events—SP:StmtStarting, SP:StmtCompleted, SQL:StmtStarting, SQL:StmtCompleted—for individual statements inside stored procedures and batches. The behavior of more granular Starting and Completed events is different depending on whether you're tracing an SQL Server 2000 machine or an SQL Server 7.0 machine, even if you're using SQL Server 2000 Profiler to trace both versions.

Regardless of whether you select to trace statements under SQL or SP, SQL Server 2000 generates only one kind of event—SQL. You can't request to trace only individual stored procedure statements or only individual batch statements. However, if you're tracing an SQL Server 7.0 machine and you include the SP:StmtStarting and SP:StmtCompleted events, you'll get the requested events only for each individual statement inside a stored procedure. If you include the SQL:StmtStarting and SQL:StmtCompleted events, you'll get the requested events only for each individual statement inside a batch.

Whether you include only the SP:StmtStarting and SP:Stmt-Completed events, only the SQL:StmtStarting and SQL:Stmt-Completed events, or all these events, you'll get the SQL:StmtStarting and SQL:StmtCompleted events for each statement inside a batch and also for each statement inside a stored procedure.

Create table t1(col1 int)
Create table t2(col1 int)
Create table t3(col1 int)
Create table t4(col1 int)
Go
Create Proc proc1
As
Select * from t1
Select * from t2
Go
Create proc proc2
As
Select * from t3
Select * from t4
Go

To demonstrate this point, I used the preceding script to create the tables t1, t2, t3, and t4 and the stored procedures proc1 and proc2 in both SQL Server 2000 and SQL Server 7.0. Then I used SQL Server 2000 Profiler to trace the following batch against both SQL Server releases:

EXEC proc1
EXEC proc2

The first trace (SP + SQL) included all possible Starting and Completed events. The second trace (SP) included all Starting and Completed events except SQL:StmtStarting and SQL:StmtCompleted. The third trace (SQL) included all Starting and Completed events except SP:StmtStarting and SP:StmtCompleted. The fourth trace (blank) included only the less granular Starting and Completed events.

Note that when tracing an SQL Server 7.0 machine, you can decide which of the more granular events to include in the trace, whereas you can trace only all or none of the more granular events when you're tracing an SQL Server 2000 machine.

I find this all-or-nothing change in behavior between SQL Server 7.0 and SQL Server 2000 surprising. If you've done tracing against SQL Server 7.0, you need to be aware of this significant change in behavior before you start tracing against SQL Server 2000. You also need to change your perspective on tracing individual batch and stored procedure statements. When tracing an SQL Server 2000 machine, treat the SP:StmtStarting and SP:StmtCompleted events as redundant, and be aware that when you trace the SQL:StmtStarting and SQL:StmtCompleted events, you actually get both stored procedure and batch statements.

Data Columns

Data columns are attributes of events. On the Data Columns tab (Figure 6.3) SQL Server 2000 lets you choose from 43 data columns as opposed to the 25 data columns available in SQL Server 7.0 Profiler. Data columns give you information about the events in your trace. SQL Server Books Online (BOL) contains details about all of Profiler's data columns and the relationship between data columns and events; not every data column is relevant for every event. Two data columns—EventClass, which shows the type of event, and SPID, which contains the server process ID—are mandatory. The other 41 data columns are optional.

Figure 6.3. Data columns tab

Some of the new data columns—such as Success, which indicates success (1) or failure (0) of a certain permission check—are relevant to the new Security Audit events. Other new data columns, such as DatabaseName and ObjectName, give you more descriptive details about the traced events. DatabaseName and ObjectName don't exist in SQL Server 7.0 Profiler, but if SQL Server 7.0 Profiler can connect to the SQL Server machine that the trace data was captured from, it tries to populate the Database ID and Object ID with the database and object names, respectively. SQL Server 2000 Profiler has separate data columns for object and database IDs and for object and database names.

Data columns generally fall into three groups: performance, informative, and event specific. Performance data columns include CPU, Duration, Reads, and Writes. Informative data columns give you information such as the start or end time of the event, database ID, login name, and database username.

Event specific data columns include EventSubClass, IntegerData, and BinaryData. I call these data columns event specific because they provide different information for different events—and no information at all for some events. For example, for the Add Audit Login to Server Role event, the EventSubClass data column shows 1 for Add and 2 for Drop. For the Audit Backup/Restore event, the EventSubClass data column shows 1 for Backup and 2 for Restore. For the Data File Auto Grow event, the IntegerData data column provides the number of pages by which the file increased, but for the Lock:Deadlock event, it provides the deadlock number (starting with 0 for the first deadlock since the server was started). For the Lock:Acquired event, the BinaryData data column contains the resource type, but for the Existing Connection event, it contains a bitmap that stores information about the session-level settings.

In short, make sure you consult BOL when you want to know what kind of information you can get from the event-specific data columns for each event. You can find this information on the Contents tab under Administrating SQL Server, Monitoring Server Performance and Activity, Monitoring with SQL Server Profiler, Monitoring with SQL Server Profiler Event categories. Each event category has a section that describes the events in that category and the data columns relevant to each event.

On the Data Columns tab, you can determine which columns you want to group the results by when you see them on the screen. You just place the desired data columns in the Groups section. If you're used to grouping events with SQL Server 7.0 Profiler, you need to be aware of some changes in SQL Server 2000 Profiler. In SQL Server 7.0 Profiler, each group gets a header that includes the values the group represents and the number of events belonging to it. For example, if you group the results by the DatabaseID column, you'll get a different header for each database ID, with the database ID itself and the number of events that belong to the database ID. You can expand the group by clicking the plus sign that appears to the left of the group's header to see the individual events: you can collapse the group by clicking the minus sign to hide the individual events. SQL Server 2000 Profiler sorts events by the data columns you specify under the Groups section rather than grouping them, so you don't get a separate caption for each group. Make sure you're selective in choosing both events and data columns to include in your trace. Including only the events and data columns you really need will reduce the amount of data that SQL Server will collect and transfer to Profiler.

Filters

Filters let you limit the events that a trace collects. On the Filters tab (Figures 6.4 and 6.5) show, you can place filters on 34 of the 43 data columns that SQL Server 2000 Profiler offers.

Figure 6.4. Filters tab: Excluding system IDs

Figure 6.5. Filters tab: Excluding results from SQL Server Agent and SQL Profiler

The data columns you can't filter are BinaryData, EventClass, EventSubClass, IntegerData, LoginSid, NestLevel, ServerName, TargetLoginSid, and TransactionID. SQL Server 2000 considerably enhances Profiler's trace-filtering capabilities by supplying powerful new event criteria options. Each filter lets you specify one or more combinations of the following options: Like, Not Like, Equals, Not Equal To, Greater Than or Equal, and Less Than or Equal. However, not all event criteria options are available with each filter. In general, textual data columns such as ApplicationName, DatabaseName, and HostName let you specify the Like and Not Like options. Numeric data columns such as DatabaseID, CPU, and SPID let you specify the Equals, Not Equal To, Greater Than or Equal, and Less Than or Equal options. Temporal data columns, which include StartTime and EndTime, let you specify only the Greater Than or Equal and Less Than or Equal options.

SQL Server 2000 Profiler also introduces the ability to specify multiple values in one filter. You can specify multiple values with the Like, Not Like, Equals, and Not Equal To event criteria options. For example, to capture events for SPIDs 52 and 54, expand the SPID filter, expand the Equals criteria option, specify 52 in the empty text box, press the Enter or semicolon (;) key, and specify 54 in the second text box. In SQL Server 7.0 Profiler, you can specify only one value for such a filter.

The checkbox Exclude system IDs lets you exclude events that reference system objects. By selecting this checkbox, you add the filter ObjectID >= 100. Because most system objects have an object ID that is less than 100, you eliminate most—but not all—references to system objects. This filter can considerably reduce the number of traced events because SQL Server itself performs a lot of activity against system objects. To efficiently examine trace output, you don't necessarily need to prefilter—place filters when you define a trace. By specifying the filters after you reopen the trace output file in Profiler, you can postfilter events that you've already collected. Prefiltering is very important in SQL Server 7.0: Without adequate prefilters, the trace output could become huge, causing SQL Server to miss some events. The consumer, which is the component responsible for collecting events from the queue, can't always clear the events fast enough to free space for new events when the queue reaches its threshold. The problem with prefiltering is that you might accidentally filter out events that are vital for your examination. For example, if you're trying to find the cause of an increasing number of deadlocks but you don't know which SPIDs are involved in the deadlocks and which application issues the code that causes the deadlocks, prefiltering on certain process IDs or certain applications might exclude the events that you need to examine.

In SQL Server 2000, prefiltering is still important for reducing the size of the trace output and for reducing the load on the server, but you can be more lenient with filters. The trace architecture is much more efficient and stable. So as a rule, filter out what you're certain you don't need, but if you're not sure, don't specify a filter. By reopening the trace file and using post-filters, you can examine the trace output more dynamically.


Templates

SQL Server 2000 Profiler's templates let you recycle previously saved trace definitions when you create new traces. They replace the Profiler Trace Wizard, available with SQL Server 7.0's Enterprise Manager, and the Trace Definitions, available with the SQL Server 7.0 Profiler. A template includes a certain combination of events, data columns, and filters. SQL Server provides nine predefined templates. For example, you can use the SQLProfiler TSQL_Replay template, which includes the events and data columns necessary for replaying traces, to keep from having to memorize and specify the exact list of items required for replay.

You can also create your own templates. To create a new template, select New, Trace Template from the File menu. A tabbed dialog box comes up in which you can choose your events, data columns, and filters. When you're done, click the Save As button on the General tab to save your template. Alternatively, you can select File, Save As, Trace Template to create a template based on a trace that you create. This trace can be active or stopped—or even a reopened trace file or table. Before your template will appear in the list of available template names, you have to save it to \Program Files\Microsoft SQL Server\80\Tools\Templates\SQL Server Profiler. To use a template when creating a new trace, from the General tab, simply choose the trace from the list of available template names or point to the template file.

SQL Server 2000 Profiler lets you trace SQL Server's activity much more efficiently than you can with SQL Server 7.0 Profiler. The new Profiler also gives you more information and investigative capabilities than its predecessor. You can examine the activities that applications are performing against SQL Server behind the scenes, locate slow-running queries, determine the cause of deadlocks, and more. By investing the time to master this tool, you can fully exploit its capabilities to fine-tune your SQL Server system.


Defining Your Own Trace Using Stored Procedures

SQL Server 2000 divides the tracing functionality into two separate but connected components. On the server side is the SQL Trace facility, which manages queues of events. Another thread, called a consumer thread, reads events from the queues and filters the events before it sends them to the requesting process. Events are the main unit of activity for tracing, and an event can be anything that happens inside SQL Server or between SQL Server and a client.

The other component is Profiler. You incur more overhead when you use Profiler to define and capture the events. The server-side consumer thread that captures the requested events returns OLE DB rowsets to a client-side consumer thread: then Profiler must display the rowsets' data appropriately. To reduce the amount of data sent from server to client, each row contains a minimum amount of information—an event code, a data-column code, and a value. For example, suppose through Profiler a user requests a trace of the text (or name) of each stored procedure when it finishes, the time it finishes, how much time it took to execute, and the requester's login ID. The server-side trace sends multiple rows to the client every time a stored procedure finishes. Each row contains the event code (43 for Stored Procedure Completed) and one of the requested data column codes. One row exists for the procedure name, one row for the end time, one row for the execution duration, one row for the user's login name, and so forth. Other rows exist for data columns the user didn't explicitly request because every trace returns a set of default data columns. Profiler must read these columns and pivot them to display one row for each event.


Defining a Server-Side Trace

All the events that are available through Profiler are also available through a server-side trace. For example, creating a table is an event, executing a stored procedure is an event, and sending a T-SQL batch from a client to SQL Server is an event.

You can use a set of stored procedures that Microsoft provides with SQL Server 2000 to define which events you want to trace, what data to collect, and where to save the collected information. You can specify event filters to reduce the amount of information the trace collects and stores, as you can when you use Profiler. However, when you define server-side traces, you must specify numbers instead of Profiler's meaningful names for the events and data columns.

To start a particular trace programmatically, you use the sp_trace_ [image] internal system procedures to define a server-side trace (Table 6.1). Although these procedures start with the characters sp_, like most system procedures, they aren't written in T-SQL. If you look at the definition of these procedures through Enterprise Manager or sp_helptext, SQL Server informs you that the procedures are internal. They have the special prefix sp_, so you can invoke them from any database. You can include the calls to these procedures in any SQL Server stored procedure or batch. To use client programs such as Query Analyzer to define a server-side trace, you must know only four (or optionally five) procedures.

The trickiest part of setting up a server-side trace comes when you define the events and data columns: You must use their internal ID numbers. Fortunately, the sp_trace_setevent article in SQL Server 2000 Books Online (BOL) that documents the sp_trace_setevent procedure provides the internal ID number for each event and each data column.

DECLARE @RC int, @TraceID int, @ON bit
SET @ON = 1
EXEC @RC = sp_trace_create @TraceID OUTPUT, 0, N'C:\TraceSELECT'
SELECT 'RC'= @RC, 'TraceID'= @TraceID

-- Statement completed event, start time column.
EXEC @RC = sp_trace_setevent @TraceID, 41, 14, @ON
-- Statement completed event, database ID column.
EXEC @RC = sp_trace_setevent @TraceID, 41, 3, @ON
-- Statement completed event, text column.
EXEC @RC = sp_trace_setevent @TraceID, 41, 1, @ON
-- Statement completed event, duration column.
EXEC @RC = sp_trace_setevent @TraceID, 41, 13, @ON

-- Login event, start column.
EXEC @RC = sp_trace_setevent @TraceID, 14, 14, @ON

-- Set filter for column 1 (text), data contains SELECT.
EXEC @RC = sp_trace_setfilter @TraceID, 1, 1, 6, N'%select%'

-- Start trace (status 1 = start).
EXEC @RC = sp_trace_setstatus @TraceID, 1

This code creates a trace that writes events to the C:\traceselect file. Note that I supplied a filename without a suffix. SQL Server adds the suffix .trc when it creates this file. (If I specified a filename with a suffix—for example, traceselect.trc—SQL Server would create the traceselect.trc.trc file.) If the file exists, you can't create the trace and the call to sp_trace_setstatus returns an error code of 12. The trace captures the start time, duration, name, and database of each Statement Completed event (Event ID 41), but the filter restricts the trace so that it captures only events that have SELECT in the name data column.

The parameters of these trace-stored procedures must have the correct data type. For example, the filename parameter for sp_trace_create has the nvarchar data type, so you must supply the filename as a Unicode constant with the n prefix. For the sp_trace_setevent procedure, the last argument, which sets data tracing for the event and the columns specified in the second and third parameters to ON or OFF, must have the bit data type. Because we can't use a CAST or CONVERT statement in the procedure call to force a constant of 1 to have the bit data type, we defined a variable to hold the value, and we passed that variable to every call. To modify this trace without stopping and redefining it, you need to pass a bit value of 0 to stop capturing a particular data column. Finally, for the sp_trace_setfilter procedure, you must explicitly pass the character arguments, such as the pattern string N'%select %', as Unicode constants.

Table 6.1. Procedures needed to define a server-side trace
Name of the procedure | Purpose
Sp_trace_create | Sets up the trace and configures a file to keep the captured events; returns a trace ID number that other procedures then use to further define and manipulate the trace
Sp_trace_setevent | Sets up event capture (once for every data column for every event the trace captures)
Sp_trace_setfilter | Sets up filter definitions (once for each filter you define on an event data column)
Sp_trace_setstatus | Starts, stops, and removes a trace
Sp_trace_generateevent | Generates user-defined events (this is optional)

Let's take a closer look at the procedure that creates the trace.

sp_trace_create [ @traceid = ] trace_id OUTPUT
         , [ @options = ] option_value
         , [ @tracefile = ] 'trace_file'
         [ , [ @maxfilesize = ] max_file_size ]
          [ , [ @stoptime = ] 'stop_time' ]

Note that this trace has two additional parameters that I didn't supply in the code shown before: a maximum file size and a stop time. The stop time is applicable only after the trace has actually started; later in the code, the sp_trace_setstatus procedure starts the trace. The @traceid parameter returns an integer that we must use if we want to modify the trace, stop or restart it, or look at its properties. The code returns the trace ID's value for informational purposes only, so we'll have the ID if we need to refer to this trace while it runs. Sp_trace_create's second parameter, @options, lets us specify one or more trace options (Table 6.2).

A value of 1 tells the trace to produce a rowset and send it to Profiler. You can't use this option value if you capture to a server-side file. Typically, only Profiler-created traces have this option value. Traces that use the sp_trace_create stored procedure should never have an option value of 1, because this value is reserved for Profiler-defined traces.

Because the option_value parameter is a bitmap, you can combine values by adding them together. For example, if you want a trace that enables file rollover and shuts down SQL Server if SQL Server can't write to the trace file, the option value is 6 (4 + 2). Note that all option values cannot be combined with all others (e.g., option value 8 by definition doesn't combine with any other option value).

Constantly running some traces can be useful, but be careful about the amount of information you capture. Trace files can grow quite large if they run the entire time that SQL Server runs. To configure a trace to begin automatically every time SQL Server starts, you can put the trace definition into a stored procedure, then use sp_procoption to give that procedure the startup property. The following code illustrates how to create a procedure that defines the trace shown before and marks the procedure to start automatically.

Table 6.2. Possible @options values that can be supplied to sp_trace_create
Option name | Option value | Description
TRACE_PRODUCE_ROWSET | 1 | Specifies that the trace will produce a rowset.
TRACE_FILE_ROLLOVER | 2 | Specifies that when the max_file_size is reached, the current trace file is closed and a new file is created. All new records will be written to the new file. The new file will have the same name as the previous file, but an integer will be appended to indicate its sequence. For example, if the original trace file is named filename.trc, the next trace file is named filename_1.trc, the following trace file is filename_2.trc, and so on. As more rollover trace files are created, the integer value appended to the filename increases sequentially. SQL Server uses the default value of max_file_size (5 MB) if this option is specified without a value for max_file_size.
SHUTDOWN_ON_ERROR | 4 | Specifies that if the trace cannot be written to the file for whatever reason, SQL Server shuts down. This option is useful when performing security audit traces.
TRACE_PRODUCE_BLACKBOX | 8 | Specifies that a record of the last 5 MB of trace information produced by the server will be saved by the server. Incompatible with all other options.

USE master
GO
CREATE PROCEDURE  USP_Trace
AS
DECLARE @TraceID int, @ON bit
SET @ON = 1
EXEC sp_trace_create @TraceID OUTPUT, 0, N'C:\TraceSELECT'

EXEC sp_trace_setevent @TraceID, 41, 14, @ON
EXEC sp_trace_setevent @TraceID, 41, 3, @ON
EXEC sp_trace_setevent @TraceID, 41, 1, @ON
EXEC sp_trace_setevent @TraceID, 41, 13, @ON
EXEC sp_trace_setevent @TraceID, 14, 14, @ON

EXEC sp_trace_setfilter @TraceID, 1, 1, 6, N'%select%'

-- Start trace (status 1 = start).
EXEC sp_trace_setstatus @TraceID, 1
RETURN
GO
-- After creating the procedure, mark it as a startup procedure.
EXEC sp_procoption TraceSELECT, 'startup', true
GO

SQL Server 2000 provides several functions you can use to inspect the status of your traces. The most useful is probably fn_trace_getinfo(), which reports all the parameters that the sp_trace_create procedure specifies, as well as the trace's status (1 = running, 0 = stopped). Here's an example of how to use fn_trace_getinfo():

SELECT * FROM :: fn_trace_getinfo(default)

You could supply a particular trace ID instead of the word "default," but if you don't know the trace ID, you can use this form of the statement to see all the traces. To view the trace file's contents and use Profiler to see all the events you've captured, you must remove the trace definition from SQL Server. Stopping the trace isn't enough to let you open the trace file, but you must stop the trace before you remove it. If you try to remove the trace before you stop it, nothing will happen—you won't even get an error message! The following code shows you how to stop and remove a trace with an ID of 1. If you don't know the ID, you can examine the results from the fn_trace_getinfo() function.

-- Stop trace (status 0 = stop).
-- The function fn_trace_getinfo() will still show the trace and
--    you can restart it if you set the status back to 1.
EXEC  sp_trace_setstatus 1, 0

-- Remove trace (status 2 = remove).
-- The function fn_trace_getinfo() will no longer show the trace
--    and you can't restart it.
EXEC  sp_trace_setstatus 1, 2

Fn_trace_geteventinfo() shows you the events and data columns that a particular trace captures, but the function returns the data with the event and data column IDs instead of a name or explanation, so you must track down their meanings. Fn_trace_getfilterinfo() returns information about a particular trace's filters.

You might have noticed that none of the supplied system procedures lets you capture your server-side trace data to a table. This apparent omission is actually a benefit because writing to a table isn't as quick as writing to a file; no matter how busy your server is, having the trace-capture operations not lose any events is essential. Writing to a server-side file uses the same mechanism that SQL Server uses to write to the transaction log, and the trace opens the file only for sequential writing. If you want to capture the data as events occur, table writes would need to happen a row at a time.

If you want to view your trace data in a table, you can copy the data from the trace file to a table after the trace collects the data. Having the trace data in a table lets you use SQL queries to analyze the data. To copy the captured trace data to a table, you can use the table-valued function fn_trace_gettable() in the SELECT INTO statement's FROM clause. This function returns a rowset with one row for each event and a column for every data column that you can trace. You'll probably be interested in only a few columns. Make sure you stop and remove the trace definition before you try to copy the trace data to a table. The following example saves the trace data from the file C:\traceselect.trc to a table called TraceDataTable.

SELECT *
INTO TraceDataTable
FROM ::fn_trace_gettable(N'C:\TraceSELECT.trc', -1)

When you store trace data in a table, the EventClass column stores the event type as an integer. The event names that correspond to each integer are in BOL along with the documentation for sp_trace_setevent, but having those event names in a SQL Server table would be for doing a table lookup.

Server-side traces are wonderfully useful in all kinds of situations and much more efficient than client-side tracing with Profiler. If you avoid server-side traces because you think writing all the procedure calls and looking up all the necessary event and data column ID information is too hard, take heart. You can use Profiler to define all the events, data columns, and filters you want, then use the Script Trace option on the File menu to produce the calls to the stored procedures that correspond to the trace elements you select. The only change you need to make is to provide a filename to hold the captured data. Then you can run the newly created script through Query Analyzer to run your server-side trace.

Caution

SQL Server Profiler is an excellent tool, but choose your counters in an effective manner. Choosing all the counters/lots of counters will give you a lot of information from which you will find it difficult to debug the cause of a problem. Also, monitoring a lot of counters incurs CPU overhead, since starting a trace always causes additional processes to be invoked that require CPU power to perform. By taking this CPU power and using it for tracing, concurrent database tasks have less CPU power available to process.


Known Bug in SQL Server 2000 Profiler

There is a bug in SQL Server 2000 that causes the Profiler to return incomplete deadlock information.

Symptoms

When an analysis is viewed, the deadlock information is incomplete. The SPID information is incorrect for one connection, a deadlock chain is missing, and the SQL that caused one of the transactions to deadlock is missing. One major problem is a change in the IntegerData column value for the Deadlock and DeadlockChain events that are associated with one another. Other information may be missing or incorrect.

Cause

The SQL Server 2000 profiling API is not correctly trapping deadlock information. This is a known bug in SQL Server 2000.

Resolution

During an SQL Server 2000 analysis, Coefficient makes an attempt to reconcile the problem in the _trace_spProcessDeadlockEvent stored procedure (_trace_spProcessDeadlockEvent2000.sql). The major problem of differences in the IntegerData values of grouped deadlock events is corrected under normal circumstances. However, missing information like SPIDs, SQL statements, and so on cannot be corrected. This has been fixed in Service Pack 1.


Index Tuning Wizard

The Index Tuning Wizard is a tool that accepts a workload of activity against the SQL Server database and makes recommendations about creating appropriate indexes for the workload. The input workload can be a trace file or a table, among other options. One of the wizard's most important new features is that it can make recommendations about creating indexed views, which are new in SQL Server 2000. The wizard can also recommend how to create the views themselves.

The Index Tuning Wizard in SQL Server 2000 lets you choose from three possible tuning modes: fast, medium, and thorough. These modes give you more flexibility in controlling how much load the wizard will put on the server, how long the tuning process will take, and how efficient the wizard's recommendations will be.

The wizard also displays the number of actual rows in each table you're tuning and lets you specify the number of projected rows. Then, if you're tuning a system in which some tables don't reflect the number of rows that you expect in the production system, you can specify the number of projected rows and the wizard can give you more accurate recommendations.

Another new and important feature is that you can fully automate the index-tuning process by using the command line utility itwiz.exe. The itwiz.exe utility accepts command line arguments in which you can specify all the tuning options you're interested in, and it runs in batch mode. You can even use an SQLServerAgent job to schedule the invocation of itwiz.exe on a recurring basis, say once a month, and examine the index recommendations saved in the .sql output script file.

The Index Tuning Wizard has some limitations. It does not recommend indexes on tables referenced by cross-database queries that do not exist in the currently selected database, system tables, or PRIMARY KEY constraints and unique indexes.

  • It is limited to a maximum of 32,767 tunable queries in a workload. Additional queries in the workload are not considered. Additionally, queries with quoted identifiers are not considered for tuning.

  • It gathers statistics by sampling the data. Consequently, successive executions of the wizard on the same workload may result in variations in the indexes recommended as well as the improvements that result from implementing the recommendation.

  • It cannot be used to select or create indexes and statistics in databases on SQL Server version 6.5 or earlier.

  • It does not give an error when saving an SQL Script to a disk with insufficient available space.

  • It can consume significant CPU and memory resources during analysis. It is recommended that tuning be performed against a test version of the production server rather than the production server. Additionally, the wizard should be run on a separate computer from the computer running an instance of SQL Server.

  • It may not make index suggestions if there is not enough data in the tables being sampled or the suggested indexes do not offer enough projected improvement in query performance over existing indexes.

The queries in the workload are analyzed in the security context of the user who invokes the Index Tuning Wizard. The user must be a member of the sysadmin fixed server role.

To reduce the execution time of the Index Tuning Wizard:

  • Ensure that "Perform thorough analysis" is not selected in the Select Server and Database dialog box. Performing a thorough analysis causes the Index Tuning Wizard to perform an exhaustive analysis of the queries, resulting in a longer execution time. However, selecting this option can result in a greater overall improvement in the performance of the tuned workload.

  • Tune only a subset of the tables in the database.

  • Reduce the size of the workload file.

The Index Tuning Wizard does not recommend that any indexes be dropped if the "Keep all existing indexes" option is selected. Only new indexes are recommended, if appropriate. Clearing this option can result in a greater overall improvement in the performance of the workload. Additionally, the Index Tuning Wizard does not recommend dropping indexes on PRIMARY KEY constraints or UNIQUE indexes. However, it may drop or replace a clustered index that is not unique or currently created on a PRIMARY KEY constraint.

The Index Tuning Wizard includes any index hint or query hint in the final recommendation, even if the index is not optimal for the table. Indexes on other tables referenced in the query may be proposed and recommended; however, all indexes specified as hints will always be part of the final recommendation. Hints can prevent the Index Tuning Wizard from choosing a better execution plan. Consider removing any index hint from queries before analyzing the workload.


Analyzing Index Tuning Wizard Output

The most important output from the Index Tuning Wizard is a set of recommended indexes. The Index Recommendations screen displays a list of these indexes, indicating the assigned index name, the table or view on which the index is defined, the order and collation (ASC or DESC) of columns in the index, whether the index is clustered, and whether the index exists. The Index Recommendations screen also displays which indexes, if any, were recommended to be dropped by the Index Tuning Wizard. The wizard also produces an estimate of the expected improvement in the execution time of the workload compared to the existing configuration. The Index Tuning Wizard uses the optimizer component of the query processor to project this estimate. However, because the optimizer's projection is based on statistical information, the actual change in performance may be different from the projected estimate.

Index Tuning Wizard recommendations are augmented by a range of reports that provide further analysis of the recommendations and their quantitative impact. These reports affect the decision about whether the recommendations should be accepted or rejected. All the reports can be saved into files for further analysis. In the Index Recommendations dialog box, click Analysis to view these report options:

  • Index Usage Report (recommended or current configuration) presents information about the expected relative usage of the recommended or current indexes and their estimated sizes.

  • Query Cost Report indicates to the user the estimated reduction or increase in the cost for execution of the 100 most expensive Transact-SQL statements in the workload file, if the recommended configuration is accepted.

  • Table Analysis Report provides information about the relative hits of the queries in the workload by tables in the database.

  • View-Table Relations Report provides information about the tables referenced by each indexed view in the recommendation.

  • Query-Index Relations Report provides information about the indexes and indexed views referenced by the 100 most expensive statements in the workload for the recommended or current set of indexes and indexed views.

  • Workload Analysis Report provides information about the relative frequencies of SELECT, INSERT, UPDATE, and DELETE queries and their relative impact on the total cost of the workload.

  • Tuning Summary Report provides important summary information about the execution of the Index Tuning Wizard. In particular, this report indicates the number of tables tuned, the number of new indexes and indexed views recommended, and the number of proposed indexes and indexed views to be dropped. The report indicates the total number of queries in the workload that were considered as well as the time taken for analysis by the Index Tuning Wizard.

Finally, the Index Tuning Wizard enables the scheduling of a task to update the existing index configuration. The index creation and alteration step can be initiated immediately or can be scheduled to occur at a specific date and time. In addition, a script to perform the index update can be created. This is particularly useful because the index recommendations can be ported from the test computer to production computers by using the script. Furthermore, the script makes it easy to identify the indexes and indexed views that will be dropped if the recommendations of the Index Tuning Wizard are accepted. Examining the script identifies two essential components of index tuning: a set of indexes and a set of statistics. Executing the recommendations to create a set of statistics is vital to harnessing the full benefits of indexing. This is because the query processor exploits statistical information during query optimization to determine whether to use an index or an indexed view.


Starting the Index Tuning Wizard

The Index Tuning Wizard can be started from the SQL Server Enterprise Manager. On the Tools pull-down menu, select Wizards, then Management. The Index Tuning Wizard is one of the wizards in the Management group. Alternatively, it can be started from SQL Profiler on the Tools tab of the Profiler menu. After the wizard obtains all the necessary user input on required parameters, it carefully begins searching the possible configurations. If the search for the index configurations is terminated during this time, the wizard returns the best available configuration that has been considered thus far. The SQL Server Query Analyzer provides another mode in which the Index Tuning Wizard can be started for a workload consisting of one or more Transact-SQL statements in a Query Analyzer buffer. This is accomplished by selecting one Transact-SQL statement in the buffer and then selecting "Perform index analysis" from the options available on the Query tab. The ability to invoke the index-tuning capability from the SQL Server Query Analyzer is useful, for example, for tuning an underperforming query in an otherwise well-tuned system.

In SQL Server 2000, index tuning can also be invoked from a command line utility called itwiz. This utility enables scripting of the index tuning process. Tuning using itwiz is similar to tuning via the Index Tuning Wizard interface with the following exceptions: (a) itwiz allows a mode in which only indexed views are considered during tuning—that is, no new indexes on tables are considered; (b) the Analysis Reports described earlier are not generated by itwiz. Instead, itwiz always generates a script file as output, which contains the statements for implementing the recommendation and additional information about the tuning process, such as the expected improvement, the storage consumed, and the time taken for analysis. This script file can be executed to implement the recommendations.


Itwiz.exe

The itwiz utility allows the Index Tuning Wizard to be executed using a command prompt utility. The Index Tuning Wizard can also be started from the SQL Server Enterprise Manager, SQL Query Analyzer, and SQL Profiler.

Syntax

itwiz
    [-?] |
    [
        -D database_name {-i workload_file | -t workload_trace_table_name}
        -o script_file_name
        [-S server_name[\instance]]
        {
            {-U login_id [-P password]}
            | –E
        }
        [-f tuning_feature_set]
        [-K keep_existing_indexes]
        [-M recommendation_quality]
        [-B storage_bound]
        [-n number_of_queries]
        [-C max_columns_in_index]
        [-T table_list_file]
        [-m minimum_improvement]
        [-F][-v]
    ]

Arguments

-? Displays usage information.

-D database_name specifies the name of the database to be tuned.

-i workload_file. Specifies the name of the workload file to use as input for tuning. The file must be in one of these formats: .trc (SQL Profiler trace file), .sql (SQL file), or .log (SQL Server 7.0 trace file).

-t workload_table_name specifies the name of a table containing the workload trace for tuning. The name is specified as [server_name].[database_name].[owner_name].table_name. The first three parameters are optional and can be omitted by marking their positions with a period (Table 6.3).

-o script_file_name specifies the name of the file to which itwiz writes the recommendation script. By default, output files are created in the current directory. The recommendation script contains the expected improvement if the recommendation is accepted.

-S server_name[\instance] specifies the computer and instance of SQL Server to connect to. If no server_name or instance is specified, itwiz connects to the default instance of SQL Server on the local computer. This option is required when executing itwiz from a remote computer on the network.

-U login_id specifies the login ID used to connect to SQL Server.

Table 6.3. Default values for the parameters
Parameter | Default value
server_name | server_name specified with -S option. If the -S option is not specified, server_name defaults to the local computer.
database_name | database_name specified with -D option
owner_name[1] | dbo
table_name | None

[1] owner_name must be dbo. If any other value is specified, execution of itwiz will fail and an error will be returned.

-P password specifies the password for the login ID. If this option is not used, itwiz prompts for a password. If this option is used without specifying a password, itwiz uses the default password (NULL).

-E uses a trusted connection instead of requesting a password.

-f tuning_feature_set specifies the features to be considered by itwiz for tuning (Table 6.4).

-K keep_existping_indexes specifies whether itwiz is allowed to propose a recommendation that requires dropping one or more existing indexes (Table 6.5).

Table 6.4. Different values for selecting features to be tuned by itwiz
Value | Description
0 | All features (default)
1 | Indexes only
2 | Indexed views only (applies only to SQL Server 2000, Enterprise and Developer editions)

Table 6.5. Values for specifying whether to keep or drop indexes
Value | Description
0 | Do not keep existing indexes.
1 | Keep all existing indexes (default).

-M recommendation_quality specifies the desired point in the running time versus quality of recommendation tradeoff. Higher values of recommendation_quality yield better quality of recommendation. Currently, recommendation_quality can be one of the values shown in Table 6.6. Fast mode currently has these restrictions: (1) no new clustered indexes are recommended; (2) no new indexed views are recommended. All existing indexes are kept (this is equivalent to specifying the -K 1 option).

-B storage_bound specifies the maximum space in megabytes that can be consumed by the recommended index set. The default storage bound is three times the current data size or the maximum available space on all attached disk drives, whichever is smaller. The current data size consists of all tables and clustered indexes.

-n number_of_queries specifies the number of queries to be tuned. By default, 200 queries are randomly chosen from the specified workload file. If number_of_queries exceeds the number of queries in the workload file, all queries are tuned.

-C max_columns_in_index specifies the maximum number of columns in indexes proposed by itwiz. The default value is 16, the maximum value allowed by SQL Server.

-T table_list_file specifies the name of a file containing a list of tables to be tuned. Each table listed within the file should begin on a new line. Table names can be qualified by a user name. Optionally, to invoke the table-scaling feature, the name of a table can be followed by a number indicating the projected number of rows in the table. The table-scaling feature enables studying recommended indexes on smaller-scale sample databases. A reasonable size (several percent, thousands of rows per table) should be used for the smaller sample database, otherwise the scaled data distribution histograms may be inaccurate and the set of recommended indexes for the sample database may be different from the index recommended for the full scale database.This is the file format for table_list_file:

Table 6.6. Analysis mode values
Value | Description
0 | Fast mode
1 | Medium mode (default)
2 | Thorough analysis mode

[owner.]table [number_of_rows]
[owner.]table [number_of_rows]
  ...   ...      ...

If the -T option is omitted, all user tables in the specified database are considered for tuning.

-m minimum_improvement—if the -m option is specified, itwiz does not recommend any changes in the index configuration unless the expected improvement in performance for the selected workload is at least minimum_improvement%. If all queries are not considered for tuning (see option -n), the queries not selected are not considered when the improvement is evaluated.

-F permits itwiz to overwrite an existing output file. In the event that an output file with the same name already exists and -F is not specified, itwiz returns an error.

-v enables verbose output from itwiz. If -v is not specified, itwiz directs only abbreviated information to the screen during execution.


Summary

In this chapter we focused on the changes in the feature sets of the SQL Server Profiler and Index Tuning Wizard and what we can achieve with these tools. The Profiler has a very rich feature set that can help to debug the application but it also has serious performance issues in the application and the database server. The Index Tuning Wizard can recommend valuable advice for index creation based on the sample load.


Chapter 7. Replication Enhancements

Microsoft SQL Server 2000 replication is a set of technologies for copying and distributing data and database objects from one database to another and then synchronizing between databases for consistency.

Using replication, data can be distributed to different locations, to remote or mobile users over a local area network, using a dial-up connection, and over the Internet. Replication also permits enhancement of application performance, physical separation of data based on how it is used (for example, for online transaction processing (OLTP) and for decision support systems), and distribution of database processing across multiple servers.

Replication offers various benefits depending on the type of replication and the options you choose; the common benefit of SQL Server 2000 replication is the availability of data when and where it is needed. Other benefits include the following.

  • Multiple sites can keep copies of the same data, useful when multiple sites need to read the same data or need separate servers for reporting applications.

  • OLTP applications are separated from read-intensive applications such as online analytical processing (OLAP) databases, data marts, or data warehouses.

  • Users can work with copies of data while disconnected and then propagate changes they make to other databases when they are connected.

  • Scale out of data to be browsed, such as browsing data using Web-based applications.

  • Aggregate read performance increases.

  • Data are brought closer to individuals or groups, helping to reduce conflicts based on multiple user data modifications and queries because data can be distributed throughout the network, and you can partition data based on the needs of different business units or users.

  • Replication can be part of a customized standby server strategy that might also include log shipping and failover clustering, which provide copies of data in case of server failure.


Types of Replication

Microsoft SQL Server 2000 provides the following types of replication that you can use in your distributed applications: (1) snapshot replication, (2) transactional replication, and (3) merge replication.

Each type provides different capabilities, depending on your application, and different levels of ACID (atomicity, consistency, isolation, durability) properties of transactions and site autonomy. For example, merge replication allows users to work and update data autonomously, although ACID properties are not assured. Instead, when servers are reconnected, all sites in the replication topology converge to the same data values. Transactional replication maintains transactional consistency, but subscriber sites are not as autonomous as they are in merge replication because publishers and subscribers generally should be connected continuously for updates to be propagated to subscribers.

It is possible for the same application to use multiple replication types and options. Some of the data in the application may not require any updates at subscriber sites, some sets of data may require infrequent updates, with updates made at only one or a few servers, while other sets of data may need to be updated daily at multiple servers.

Which type of replication you choose for your application depends on your requirements based on distributed data factors, whether or not data will need to be updated at subscriber sites, your replication environment, and the needs and requirements of the data that will be replicated.


Replication Options

Replication options allow you to configure replication in a manner best suited to your application and environment.

  • Filtering published data involving snapshot, transactional, and merge replication. Filters permit creation of vertical and/or horizontal partitions of data that can be published as part of replication. By distributing partitions of data to different subscribers, you can minimize the amount of data sent over the network, reduce the amount of storage space required at the subscriber, customize publications and applications based on individual subscriber requirements, and reduce conflicts because the different data partitions can be sent to different subscribers.

  • Updatable subscriptions (immediate updating, queued updating) involving snapshot and transactional replication. Immediate updating and queued updating options allow users to update data at the subscriber and either propagate those updates to the publisher immediately or store the updates in a queue. Updatable subscriptions are best for replication topologies where replicated data is mostly read and occasionally updated at the subscriber when publisher, distributor, and subscriber are connected most of the time and when conflicts caused by multiple users updating the same data are infrequent.

  • Updatable subscriptions involving merge replication. Merge replication allows users to update data at the subscriber or publisher and synchronize changes continuously, on demand, or at scheduled intervals. Merge replication is well suited for topologies where replicated data is frequently updated at the subscriber even when the subscriber is disconnected from the publisher. Conflicts caused by multiple users updating the same data should be infrequent, but merge replication provides a rich set of options for handling conflicts that do occur.

  • Transforming published data involving snapshot and transactional replication. Data movement, transformation mapping, and filtering capabilities of Data Transformation Services (DTS) can be leveraged during replication. With transformable subscriptions, you can create custom partitions for snapshot and transactional publications and transform the data as it is being published with data type mappings (for example, integer to real data type), column manipulations (for example, concatenating first name and last name columns into one), string manipulations, and functions.

  • Alternate synchronization partners involving merge replication. Alternate synchronization partners allow merge subscribers to synchronize data with servers other than the publisher at which the subscription originated, permitting the subscriber to synchronize data when the original publisher is unavailable. This option is also useful for mobile subscribers that may have access to a faster or more reliable network connection with an alternate server.

  • Optimizing synchronization involving merge replication. By optimizing synchronization during merge replication, you can store more information at the publisher instead of transferring that information over the network to the subscriber. This improves synchronization performance over a slow network connection, but requires additional storage at the publisher.


New Features and Improvements

Microsoft SQL Server 2000 has the following replication improvements and new features.

Implementing Replication

SQL Server 2000 enhances snapshot replication, transactional replication, and merge replication with the following features.

  • Alternate snapshot locations provide easier and more flexible methods for applying the initial snapshot to subscribers. You can save (and compress) the snapshot files to a network location or removable medium, which can then be transferred to subscribers without using the network.

  • Attachable subscription databases permit transfer of a database with replicated data and one or more subscriptions from one subscriber to another SQL Server. After the database is attached to the new subscriber, the subscription database at the new subscriber automatically receives its own pull subscriptions to the publications at the specified publishers.

  • Schema changes on publication databases permit addition or deletion of columns on the publishing table and propagation of those changes to subscribers.

  • On-demand script execution permits posting of a general SQL script that will be executed at all subscribers.

  • Presnapshot and postsnapshot scripts permit scripts to be run before or after a snapshot is applied at the subscriber.

  • Remote agent activation permits reducing the amount of processing on the distributor or subscriber by running the distribution agent or merge agent on one computer while activating that agent from another computer. You can use remote agent activation with push or pull subscriptions.

  • Support of new SQL Server features includes user-defined functions, indexed views, new data types, and multiple instances of SQL Server.

  • ActiveX Snapshot Control makes programmatic generation of snapshots easier.

  • More snapshot scripting options support transfer of indexes, extended properties, and constraints to subscribers.

Merge Replication

Merge replication is the process of distributing data from publisher to subscribers, allowing the publisher and subscribers to make updates while connected or disconnected, and then merging the changes between sites when they are connected. Enhancements to merge replication include the following.

  • Greater parallelism of the merge agent for improved server-to-server performance.

  • Optimizations for determining data changes relevant to a partition at a subscriber.

  • Dynamic snapshots, which provide more efficient application of the initial snapshot when using dynamic filters.

  • Vertical filters for merge publications.

  • More powerful dynamic filtering with user-defined functions.

  • The ability to use alternate synchronization partners when synchronizing data. Using alternate synchronization partners, a subscriber to a merge publication can synchronize with any specified server that has the same data as the original publisher.

  • Automated management of identity ranges. In merge replication topologies where a publication contains an identity column and where new rows can be inserted at subscribers, automated management of identity ranges at the subscriber ensures that the same identity values are not assigned to rows inserted at different subscription databases, and that primary key constraint violations do not occur. This feature is also available when queued updating is used with snapshot replication or transactional replication.

  • Support for timestamp columns in published tables.

  • Improved management of the growth of merge tracking data.

  • Several new merge replication conflict resolvers, including interactive resolvers that provide a user interface for immediate, manual conflict resolution, priority based on a column value, minimum/maximum value wins, first/last change wins, additive/average value, and merge by appending different text values.

  • New options to validate permissions for a subscriber to upload changes to a publisher (check_permissions) and security enhancements including code signing of conflict resolvers included with SQL Server 2000.

  • New COM interfaces that support heterogeneous data sources as publishers within an SQL Server replication topology.

  • Validation of replicated data per subscription or on a publicationwide basis. Validation is also available through SQL Server Enterprise Manager.

  • Reinitialization to allow uploading of changes from the subscriber before the application of a new snapshot.

Transactional Replication

With transactional replication, an initial snapshot of data is applied at subscribers; then when data modifications are made at the publisher, the individual transactions are captured and propagated to subscribers. Following are enhancements to transactional replication.

  • Concurrent snapshot processing so that data modifications can continue on publishing tables while the initial snapshot is generated.

  • Improved error handling and the ability to skip specified errors and continue replication.

  • Validation of replicated data at the subscriber, including validation on vertical partitions. Validation is also available through the SQL Server Enterprise Manager.

  • Publication indexed views as tables.

  • Option to store data modifications made at the subscriber in a queue (queued updating).

  • Option to transform data as it is published to subscribers (transforming published data).

  • Ability to restore transactional replication databases without reinitializing subscriptions or disabling and reconfiguring publishing and distribution. Transactional replication can also be set up to work with log shipping, enabling failover to a warm standby server without reconfiguring replication.

Queued Updating

Queued updating allows snapshot replication and transactional replication subscribers to modify published data without requiring an active network connection to the publisher.

When you create a publication with the queued updating option enabled and a subscriber performs INSERT, UPDATE, or DELETE statements on published data, the changes are stored in a queue. The queued transactions are applied asynchronously at the publisher when network connectivity is restored.

Because the updates are propagated asynchronously to the publisher, the same data may have been updated by the publisher or by another subscriber, and conflicts can occur when applying the updates. Conflicts are detected automatically, and several options for resolving conflicts are offered.

Transforming Published Data

Transformable subscriptions (available with snapshot replication or transactional replication) leverage the data movement, transformation-mapping, and filtering capabilities of Data Transformation Services (DTS). Using transformable subscriptions in your replication topology allows you to customize and send published data based on the requirements of individual subscribers, including performing data type mappings, column manipulations, string manipulations, and use of functions as data is published.


Replication Usability

Several improvements in the SQL Server Enterprise Manager provide for easier implementation, monitoring, and administration of replication:

  • Centralized replication folder in the SQL Server Enterprise Manager tree, which organizes all subscriptions and publications on the server being administered.

  • Ability to browse for and subscribe to publications (when permission is allowed) using Windows Active Directory.

  • Ability to see multiple distributors in a single monitoring node in SQL Server Enterprise Manager.

  • Standard and advanced replication options separated in the Create Publication, Create Push Subscription, and Create Pull Subscription Wizards. You can choose to show advanced options in these wizards on the Welcome page of each wizard.

  • New wizards for creating jobs that create dynamic snapshots for merge publications that use dynamic filters (Create Dynamic Snapshot Job Wizard) and for transforming published data in snapshot replication or transactional replication (Transform Published Data Wizard).


Log Shipping

Log shipping is available only in the Enterprise Edition of SQL 2000 Server. You can use log shipping to feed transaction logs from one database to another on a constant basis. Continually backing up the transaction logs from a source database and then copying and restoring the logs to a destination database keeps the destination database synchronized with the source database, permitting a backup server and providing a way to offload query processing from the main computer (the source server) to read-only destination servers.

Do not use the monitor server as the source server, because the monitor server maintains critical information regarding the log shipping system. The monitor server should be regularly backed up. Keeping the monitor server independent is also better for performance, because monitoring adds unnecessary overhead. Also, as a source server supporting a production workload, it is most likely to fail, which would disrupt the monitoring. The source and destination servers can be on the same computer. However, in this case, SQL Server 2000 failover clustering may provide better results.

Configuring Log Shipping with the Database Maintenance Plan Wizard

To easily configure log shipping, use the Database Maintenance Plan Wizard as follows:

  1. Define how often the logs are generated, the time between a backup and a restore operation, and when a destination server is out of synchronization with a source server.

  2. Register any new servers.

  3. Create the source databases on all destination servers. When adding a destination database through the Database Maintenance Plan Wizard, you have the option of creating the databases on the destination server or using existing databases. Any existing databases must be in standby mode before you can configure them for log shipping.

  4. Specify which destination servers might assume the role of the source server.

  5. Set a restore delay. This delay defines how old a transaction log must be before it is restored. If something goes wrong on the source server, this delay provides an extra time before the corrupted log is restored onto the destination server.

  6. Create a schedule that sets the backup schedule.

Before using the Database Maintenance Plan Wizard, consider the following:

  • The user configuring log shipping must be a member of the sysadmin server role in order to have permission to modify the database to log ship.

  • You can configure log shipping on only one database at a time. If you select more than one database, the log shipping option on the wizard is disabled.

  • The login used to start the MSSQLServer and SQLServerAgent services must have access to the log shipping plan jobs, the source server, and the destination server.

  • When you use the Database Maintenance Plan Wizard to configure log shipping, you can log ship only to disks, the backup-to-tape option is not available.

Configuring Log Shipping Manually

SQL Server 2000 supports manual log shipping from an SQL Server version 7.0 Service Pack 2 (SP2) transaction log if the pending upgrade option is enabled on the computer running SP2.

To enable this option, execute the following code:

EXEC sp_dboption 'database name', 'pending upgrade', 'true'

However, when you are restoring the database after log shipping, you can recover only with the NORECOVERY option.


Summary

In this chapter we highlighted the new features and improvements provided by SQL Server 2000 in the area of replication and log shipping. Replication is a vast topic, and one chapter cannot do full justice to it or cover all the scenarios related to replication. The intent of this chapter was to list the new feature sets and the areas in which improvements have been made. Log shipping permits maintenance of a standby copy of the database and is ideal for reporting purposes.


Chapter 8. New Data Types

Like its predecessor, SQL Server 7.0, SQL Server 2000 delivers several new data types. SQL Server 7.0 introduced a whole range of Unicode data types and the GUID unique identifier. SQL Server 2000 introduces the 64-bit integer (bigint), variant (sql_variant), and table data types.


bigint

The bigint data type is the easiest to explain. It is just an integer that supports data from -263 (-9,223,372,036,854,775,808) through 263 - 1 (9,223,372,036,854,775,807). Storage size for the bigint is 8 bytes, hence the name 64-bit integer. You will want to use the new integer in support of identity columns where the tables will exceed the previous maximum integer size of 231 - 1 (2,147,483,647). The new bigint data type is supported where integer values are supported, but there is a caveat: Functions will return bigint only if the parameter expression is a bigint data type. SQL Server will not automatically promote other integer data types (such as tinyint, smallint, and int) to bigint.

In support of the new bigint data type, two new built-in functions have been introduced: BIG_COUNT and ROWCOUNT_ BIG. These functions operate in a manner similar to the COUNT function and @@ROWCOUNT variable except they return an integer value of type bigint.


sql_variant

The new sql_variant data type stores values of various supported data types except text, ntext, image, timestamp, and sql_variant. This data type may be used in columns, parameters, variables, and return values of user-defined functions, which are new features of SQL Server that I describe shortly. The sql_variant data type operates much like the variant data type in Visual Basic. It allows a single column, parameter, or variable to store data values of different data types. For example, a single sql_variant column can hold int, decimal, char, binary, and nchar values. Each sql_variant instance records a data value and associated metadata information, which includes the base data type, maximum size, scale, precision, and collation.

The key to working with an sql_variant data type is to remember to always cast the variant to the required base data element. This is easily achieved using the CAST operator:

Declare @my_var_sql_variant
Set @my_var = 3.142
DECLARE @my_chr VARCHAR(12), @my_dec DECIMAL(5,4)
SET @my_chr = CAST(@my_var AS VARCHAR(12))
SET @my_dec = CAST(@my_var AS DECIMAL(5,4))

To support the sql_variant data type, a new function called SQL_VARIANT_PROPERTY has been introduced. This function returns the base data type and other information about a sql_variant instance:

SQL_VARIANT_PROPERTY(expression, property)

The property parameter contains the name of the sql_variant property for which information is to be provided. It can take any one of the following values: BaseType, Precision, Scale, TotalBytes, Collation, and MaxLength. To demonstrate, take a look at the following code and output.

DECLARE @my_chr sql_variant, @my_dec DECIMAL(5,4)
SET @my_chr = '3.142'
SET @my_dec = 3.142
SELECT    SQL_VARIANT_PROPERTY(@my_chr,'BaseType') AS VarBaseType,
    SQL_VARIANT_PROPERTY(@my_chr,'Precision') AS VarPrecision,
    SQL_VARIANT_PROPERTY(@my_chr,'Scale') AS VarScale,
    SQL_VARIANT_PROPERTY(@my_chr,'TotalBytes') AS VarTotalBytes,
    SQL_VARIANT_PROPERTY(@my_chr,'Collation') AS VarCollation,
    SQL_VARIANT_PROPERTY(@my_chr,'MaxLength') AS VarMaxLength
UNION
SELECT    SQL_VARIANT_PROPERTY(@my_dec,'BaseType'),
    SQL_VARIANT_PROPERTY(@my_dec,'Precision'),
    SQL_VARIANT_PROPERTY(@my_dec,'Scale'),
    SQL_VARIANT_PROPERTY(@my_dec,'TotalBytes'),
    SQL_VARIANT_PROPERTY(@my_dec,'Collation'),
    SQL_VARIANT_PROPERTY(@my_dec,'MaxLength')
GO
VarBaseType VarPrecision VarScale VarTotalBytes VarCollation           VarMaxLength
----------- ------------ -------- ------------- ------------           ------------
varchar     0            0        13            SQL_Latin1_General_CP1_CI_AS  5
decimal     5            4        9             NULL                          5

The sql_variant data type becomes extremely useful in applications where this meta data information is managed by the application. An example would be an ASP application that collects information from a Web user in a form, especially when the form processing has been abstracted. Instead of saving the form value in the database as a character variable along with all the associated meta data information, all you need to do is save the value as an sql_variant. Just remember to cast the variable, before persisting it to the database:

Dim varName, varAge
varName = CStr(Request.Form("txtName"))
varAge = CInt(Request.Form("txtAge"))


Table

As the name implies, the new table data type is a local variable that is primarily used for temporary storage of a set of rows, which can also be returned as the result of a table-valued function. The table data type can be used in place of temporary tables stored in the tempdb database. Because the new table data type is managed in memory and is not persisted to disk, it provides performance benefits.

Once you define a local table, you can use it in the same fashion as a tempdb table. It does not, however, support INSERT INTO… EXEC and SELECT…INTO statements. To define a local table you have to use the usual DECLARE statement with a subsequent table definition:

DECLARE @local_variable TABLE <table_defintion>

To demonstrate, let's define and populate a local variable that contains all the rowset information about Northwind database territory data:

DECLARE @territory TABLE (territory_id INT, territory_desc NVARCHAR(50))
INSERT INTO @territory
SELECT TerritoryId, TerritoryDescription FROM Territories

Although this example is not very exciting, it does show how easy it is to use the table data type. The table data type can be useful when parsing XML documents in cooperation with the new OpenXML function. This function, which I discuss in detail later, allows you to receive an XML document and extract the information in rowset format. This rowset format can be a local table variable, which can be used for processing of the parsed data.

T-SQL Code 8.1: XML Document Sample

<ROOT>
<Region RegionId="5" RegionDescription="Canada">
   <Territories RegionId="5" TerritoryId="10001" TerritoryDescription="British
Columbia"/>
   <Territories RegionId="5" TerritoryId="10002" TerritoryDescription="Alberta"/>
   <Territories RegionId="5" TerritoryId="10003"
TerritoryDescription="Saskatchewan"/>
   <Territories RegionId="5" TerritoryId="10004" TerritoryDescription="Manitoba"/>
   <Territories RegionId="5" TerritoryId="10005" TerritoryDescription="Ontario"/>
   <Territories RegionId="5" TerritoryId="10006" TerritoryDescription="Quebec"/>
   <Territories RegionId="5" TerritoryId="10007"
TerritoryDescription="Newfoundland"/>
   <Territories RegionId="5" TerritoryId="10008" TerritoryDescription="Nova
Scotia"/>
   <Territories RegionId="5" TerritoryId="10009" TerritoryDescription="New
Brunswick"/>
   <Territories RegionId="5" TerritoryId="10010" TerritoryDescription="Prince
Edward Island"/>
</Region>
<Region RegionId="6" RegionDescription="United Kingdom">
   <Territories RegionId="6" TerritoryId="20001" TerritoryDescription="England"/>
   <Territories RegionId="6" TerritoryId="20002" TerritoryDescription="Scotland"/>
   <Territories RegionId="6" TerritoryId="20003" TerritoryDescription="Wales"/>
   <Territories RegionId="6" TerritoryId="20004" TerritoryDescription="Northern
Ireland"/>
</Region>
</ROOT>

T-SQL Code 8.2: XML Stored Procedure Parser

CREATE PROCEDURE dbo.usp_xml_regionterritory (@xmldoc NTEXT)
AS
BEGIN

    DECLARE @idoc    INT
    DECLARE @territory TABLE (
        region_id    INT,
        region_desc    NVARCHAR(50),
        territory_id     INT,
        territory_desc     NVARCHAR(50)
    )

    -- Create an internal representation of the XML document.
    EXEC sp_xml_preparedocument @idoc OUTPUT, @xmldoc

    -- SELECT statement using OpenXML rowset provider
    INSERT INTO @territory
    SELECT * FROM OpenXML (@idoc, '/ROOT/Region/Territories', 0) WITH (
        RegionId          INT          '../@RegionId',
        RegionDesc        NVARCHAR(50) '../@RegionDescription',
        TerrorityId       INT          '@TerritoryId',
        TerritoryDesc     NVARCHAR(50) '@TerritoryDescription')

    -- remove the XML document from memory
    EXEC sp_xml_removedocument @idoc

    -- Create the new regions
    INSERT INTO Region
    SELECT DISTINCT region_id, region_desc
    FROM @territory

    -- Create the new territories
    INSERT INTO Territories
    SELECT DISTINCT territory_id, territory_desc, region_id
    FROM @territory

END
GO

T-SQL Code 8.1 shows an XML document. The stored procedure shown in T-SQL Code 8.2 will receive and parse this XML document, inserting the parsed elements into the appropriate Northwind region and territory tables. One point to note about this stored procedure is that the insert operations do not take into account the possibility that updates of existing data may be required. I outline one solution to this problem in Chapter 11. In addition to replacing the need to use tempdb tables, the table data type becomes extremely useful when used in conjunction with user-defined functions, as you will see in the Chapter 9.


Summary

In this chapter, we introduced the three new data types available with SQL Server 2000 and the scenarios under which they can be used.


Chapter 9. User-Defined Functions

User-defined functions are similar to stored procedures and can be executed as such, or they can be executed in the same fashion as a system function. Like stored procedures, they accept zero or more input parameters but can return a scalar data type such as int, decimal, varchar, or sql_variant in addition to the new table data type. The defining structure of a user-defined function is similar to that of a stored procedure. The main differences are the requirements to specify the RETURNS value and to terminate the function with the RETURN statement.

A user-defined function that returns a table data type can declare an internal table variable and return that variable as its return value. These functions are known as rowset functions and can be used where table or view expressions are allowed in Transact-SQL queries. This means that a table returned by a user-defined function can be referenced in the from clause of a Transact-SQL statement, while stored procedures that return resultsets cannot.

User-defined functions that return a table can be a powerful alternative to views. While views are limited to a single select statement, user-defined functions can contain additional statements that allow more powerful logic than is possible in views.

Microsoft SQL Server 2000 supports built-in functions and user-defined functions. Built-in functions operate as defined in the Transact-SQL Reference and cannot be modified. The functions can be referenced only in Transact-SQL statements using the syntax defined in the Transact-SQL Reference. A couple of examples of these functions are the datetime functions and the aggregate functions. User-defined functions allow you to define your own Transact-SQL functions using the CREATE FUNCTION statement. User-defined functions are created using the CREATE FUNCTION statement, modified using the ALTER FUNCTION statement, and removed using the DROP FUNCTION statement. Each fully qualified user-defined function name (database_name.owner_name.function_name) must be unique.

You must have been granted CREATE FUNCTION permission to create, alter, or drop user-defined functions. Users other than the owner must be granted appropriate permissions on a function before they can use it in a Transact-SQL statement. To create or alter tables with references to user-defined functions in the CHECK constraint, DEFAULT clause, or computed column definition, you must also have REFERENCES permission on the functions.

Transact-SQL errors that cause a statement to be canceled and continue with the next statement in the module (such as triggers or stored procedures) are treated differently inside a function. In functions, such errors cause the execution of the function to stop. This in turn causes the statement that invoked the function to be canceled.


Types of User-Defined Functions

SQL Server 2000 supports two types of user-defined functions: scalar functions and table-valued functions. A user-defined function takes zero or more input parameters and returns either a scalar value or a table. A function can have a maximum of 1024 input parameters. When a parameter of the function has a default value, the keyword DEFAULT must be specified when calling the function to get the default value. This behavior is different from parameters with default values in stored procedures in which omitting the parameter also implies the default value. User-defined functions do not support output parameters.

Scalar Functions

Scalar functions return a single data value of the type defined in a RETURNS clause. All scalar data types, including bigint and sql_variant, can be used. The timestamp data type, user-defined data type, and nonscalar types, such as table or cursor, are not supported. The body of the function, defined in a BEGIN…END block, contains the series of Transact-SQL statements that return the value. The return type can be any data type except text, ntext, image, cursor, and timestamp.

Table-Valued Functions

These functions return a table. For an inline table-valued function, there is no function body; the table is the result set of a single SELECT statement. For a multistatement table-valued function, the function body, defined in a BEGIN…END block, contains the TRANSACT-SQL statements that build and insert rows into the table that will be returned.

Inline user-defined functions

Inline user-defined functions are a subset of user-defined functions that return a table. Inline functions can be used to achieve the functionality of parameterized views.

Inline user-defined functions follow these rules:

  • The RETURNS clause contains only the keyword table. The format of a return variable does not have to be defined because it is set by the format of the result set of the SELECT statement in the RETURN clause.

  • No function_body is delimited by BEGIN and END.

  • The RETURN clause contains a single SELECT statement in parentheses. The result set of the SELECT statement forms the table returned by the function. The SELECT statement used in an inline function is subject to the same restrictions as SELECT statements used in views.

Inline functions can also be used to increase the power of indexed views. The indexed view itself cannot use parameters in its WHERE clause search conditions to tailor the stored result set to specific users. You can, however, define an indexed view that stores the complete set of data that matches the view and then define an inline function over the indexed view that contains parameterized search conditions that allow users to tailor their results. If the view definition is complex, most of the work performed to build a result set involves operations such as building aggregates or joining several tables when the clustered index is created on the view. If you then create an inline function that references the view, the function can apply the user's parameterized filters to pull specific rows from the result set that was built by the CREATE INDEX statement. The complex aggregations and joins are done once, at CREATE INDEX time, and all subsequent queries referencing the inline function filter rows from the simplified, stored result set.

Inline user-defined functions can be used to support parameters in the search conditions specified in the WHERE clause. This is an example of a function that allows users to specify the region in their SELECT statement

CREATE FUNCTION fn_CustomerNamesInRegion
                 ( @RegionParameter nvarchar(30) )
RETURNS table
AS
RETURN (
        SELECT CustomerID, CompanyName
        FROM Northwind.dbo.Customers
        WHERE Region = @RegionParameter
       )
GO
-- Example of calling the function for a specific region
SELECT *
FROM fn_CustomerNamesInRegion(N'WA')
GO
User-defined functions that return a table data type

User-defined functions that return a table can be powerful alternatives to views. A user-defined function that returns a table can be used where table or view expressions are allowed in Transact-SQL queries. While views are limited to a single SELECT statement, user-defined functions can contain additional statements that allow more powerful logic than is possible in views.

A user-defined function that returns a table can also replace stored procedures that return a single result set. The table returned by a user-defined function can be referenced in the FROM clause of a Transact-SQL statement, but stored procedures that return result sets cannot.

In a user-defined function that returns a table, the RETURNS clause defines a local return variable name for the table returned by the function. The RETURNS clause also defines the format of the table. The scope of the local return variable name is local within the function. The Transact-SQL statements in the function body build and insert rows into the return variable defined by the RETURNS clause. When a RETURN statement is executed, the rows inserted into the variable are returned as the tabular output of the function. The RETURN statement cannot have an argument.

No Transact-SQL statements in a function that returns a table can return a result set directly to a user. The only information the function can return to the user is the table returned by the function.

The statements in a BEGIN…END block cannot have any side effects. Function side effects are any permanent changes to the state of a resource that has a scope outside the function such as a modification to a database table. The only changes that can be made by the statements in the function are changes to objects local to the function, such as local cursors or variables. Modifications to database tables, operations on cursors that are not local to the function, sending e-mail, attempting a catalog modification, and generating a result set that is returned to the user are examples of actions that cannot be performed in a function.

The following types of statements are valid in a function.

  • DECLARE statements that are used to define data variables and cursors that are local to the function.

  • Assignments of values to objects local to the function, such as using SET to assign values to scalar and table local variables.

  • Cursor operations that reference local cursors that are declared, opened, closed, and deallocated in the function. FETCH statements that return data to the client are not allowed. Only FETCH statements that assign values to local variables using the INTO clause are allowed.

  • Control-of-flow statements.

  • SELECT statements containing select lists with expressions that assign values to variables that are local to the function.

  • UPDATE, INSERT, and DELETE statements modifying table variables that are local to the function.

  • EXECUTE statements calling an extended stored procedure.

The number of times that a function specified in a query is actually executed can vary between execution plans built by the optimizer. An example is a function invoked by a subquery in a WHERE clause. The number of times the subquery and its function is executed can vary with different access paths chosen by the optimizer.

Built-in functions that can return different data on each call are not allowed in user-defined functions. The built-in functions not allowed in user-defined functions follow:

@@CONNECTIONS | @@PACK_SENT | GETDATE
@@CPU_BUSY | @@PACKET_ERRORS | GetUTCDate
@@IDLE | @@TIMETICKS | NEWID
@@IO_BUSY | @@TOTAL_ERRORS | RAND
@@MAX_CONNECTIONS | @@TOTAL_READ | TEXTPTR
@@PACK_RECEIVED | @@TOTAL_WRITE |

Schema-bound functions

CREATE FUNCTION supports a SCHEMABINDING clause that binds the function to the schema of any objects it references, such as tables, views, and other user-defined functions. An attempt to alter or drop any object referenced by a schema-bound function fails.

The following conditions must be met before you can specify SCHEMABINDING in CREATE FUNCTION.

  • All views and user-defined functions referenced by the function must be schema-bound.

  • All objects referenced by the function must be in the same database as the function. The objects must be referenced using either one-part or two-part names.

  • You must have REFERENCES permission on all objects (tables, views, and user-defined functions) referenced in the function.

  • You can use ALTER FUNCTION to remove the schema binding. The ALTER FUNCTION statement should redefine the function without specifying WITH SCHEMABINDING.

Calling user-defined functions

When calling a scalar user-defined function, you must supply at least a two-part name:

SELECT *, MyUser.MyScalarFunction()
FROM MyTable

Table-valued functions can be called by using a one-part name:

SELECT *
FROM MyTableFunction()

However, when you call built-in functions that return a table, you must add the prefix :: to the name of the function:

SELECT * FROM ::fn_helpcollations()


Obtaining Information About Functions

Two catalog objects report information about user-defined functions: sp_help reports information about user-defined functions; sp_helptext reports the source of user-defined functions. Three information schema views report information about user-defined functions: ROUTINES, PARAMETERS, and ROUTINE_COLUMNS. These information schema views also report information for stored procedures.


Summary

In this chapter we explored the functionality provided by the user-defined functions. This capability has existed in Oracle database for quite some time and its addition to the feature set of SQL Server 2000 broadens the capabilities of the Transact-SQL language. In Chapter 27, I provide another example of a user-defined function that can be used to get the same functionality as the GREATEST function in Oracle.


Chapter 10. Indexed Views

A view can be thought of as either a virtual table or a stored query. The data accessible through a view is not stored; rather, the SELECT statement forming the view is persisted. A view forms a virtual table that can be referenced in the same manner as a table. Indexed views are an extension to this definition that greatly improves the performance of views of the type normally found in data warehouses or other decision support systems.

Relational Database Management Systems (RDBMS) have supported the ability to create virtual tables known as views that serve these main purposes: to provide a security mechanism that restricts users to a certain subset of data in one or more base tables; to provide a mechanism that allows developers to customize how users can logically view the data stored in base tables. With SQL Server 2000, the functionality of SQL Server views has been expanded to provide system performance benefits. It is possible to create a unique clustered index on a view, as well as nonclustered indexes, to improve data access performance on the most complex queries. In SQL Server 2000, a view that has a unique clustered index is referred to as an indexed view. (Note: Indexed views are a feature of only the Enterprise and Developer editions of SQL Server 2000.)

From the DBMS perspective, a view is a description of the data (a meta data). When a typical view is created, the meta data is defined by encapsulating a SELECT statement that defines a result set to be represented as a virtual table. When a view is referenced in the FROM clause of another query, this meta data is retrieved from the system catalog and expanded in place of the view's reference. After view expansion, the query optimizer compiles a single execution plan for the executing query.

In the case of a nonindexed view, the view is materialized at runtime. Any computations such as joins or aggregations are done during query execution for each query referencing the view. (The view does not always need to be fully materialized. The query can contain additional predicates, joins, or aggregations that can be applied to the tables and views referenced in the view, which eliminates the need for full materialization.) After a unique clustered index is created on the view, the view's result set is materialized immediately and persisted in physical storage in the database, saving the overhead of performing this costly operation at execution time.

The indexed view can be used in a query execution in two ways. The query can reference the indexed view directly, or more important, the query optimizer can select the view if it determines that the view can be substituted for some or all of the query and it is the low-cost query plan. In the second case, the indexed view is used instead of the underlying tables and their ordinary indexes. The view does not need to be referenced in the query for the query optimizer to use it during query execution. This allows existing applications to benefit from the newly created indexed views without changing those applications.


Performance Gains from Indexed Views

Using indexes to improve query performance is not a new concept; however, indexed views provide additional performance benefits that cannot be achieved using standard indexes. Indexed views can increase query performance in the following ways:

  • Aggregations can be precomputed and stored in the index to minimize expensive computations during query execution.

  • Tables can be prejoined and the resulting data set stored.

  • Combinations of joins or aggregations can be stored.

Secondary, nonclustered indexes on views can provide additional query performance. Similar to secondary indexes on tables, secondary indexes on views may provide more options for the query optimizer to choose from during the compilation process. For example, if the query includes columns not covered by the clustered index, the optimizer can choose one or more secondary indexes in the plan and avoid a time-consuming full scan of the indexed view or base tables.

Adding indexes to the schema increases the overhead on the database because the indexes will require ongoing maintenance. Careful consideration should be given to finding the right balance of indexes and maintenance overhead.

When an indexed view is created, SQL Server must keep the data stored for the indexed view synchronized with the data of the tables on which the view is created. This means that any insert, update, or delete to the underlying tables results in the same changes to the data of the indexed view. The key performance point that needs to be stressed for indexed views is that "heavy query" = "performance gain" and "heavy modify statements" = "performance loss."


Benefits of Using Indexed Views

Analyze your database workload before implementing indexed views. Use your knowledge of the queries as well as various tools (for example SQL Profiler) to identify the queries that can benefit from indexed views. Frequently occurring aggregations and joins are the best candidates for indexed views.

Not all queries benefit from indexed views. Similar to ordinary indexes, if the indexed views are not used, there is no benefit. In this case, not only are performance gains not realized, but the additional costs of disk space, maintenance, and optimization are incurred. However, when indexed views are used, they can provide significant improvements (by orders of magnitude) in data access. This is because the query optimizer uses the precomputed results stored in the indexed view, which substantially reduces the cost of the query execution.

The query optimizer considers indexed views only for queries with nontrivial cost, thus avoiding situations where trying to match various indexed views during the query optimization costs more than the savings achieved by the indexed view usage. Indexed views are rarely used in queries with a cost of less than 1.

Applications that benefit from the implementation of indexed views include decision support workloads, data marts, online analytical processing (OLAP) stores and sources, and data mining workloads. From the query type and pattern point of view, the benefiting applications can be characterized as those containing joins and aggregations of large tables, repeated patterns of queries, repeated aggregations on the same or overlapping sets of columns, repeated joins of the same tables on the same keys, and combinations of joins and aggregations. Online transaction processing (OLTP) systems with many writes or databases with frequent updates, may not be able to take advantage of the indexed views because of the increased maintenance cost associated with updating both the view and underlying base tables.


Query Optimizer

The SQL Server query optimizer automatically determines when an indexed view can be used for a given query execution. The view does not need to be referenced directly in the query for the optimizer to use it in the query execution plan. Therefore, existing applications may take advantage of the indexed views without any changes to the application itself; only the indexed views have to be created.

The query optimizer considers several conditions to determine whether an indexed view can cover a portion or all of the query.

  • The tables in the query FROM clause must be a superset of the tables in the indexed view FROM clause.

  • The join conditions in the query must be a superset of the join conditions in the view.

  • The aggregate columns in the query must be a subset of the aggregate columns in the view.

  • All expressions in the query select list must be derivable from the view select list or from the tables not included in the view definition.

  • The query search condition predicates must be a superset of the search condition predicates in the view definition. Each conjunct in the view search predicate must appear in the same form as a conjunct in the query search predicate.

  • All columns in the query search condition predicates that belong to tables in the view definition must appear in one or more of the following: the same predicate in the view definition; a GROUP BY list; the view SELECT list if there is no GROUP BY.

These conditions correspond to a single FROM clause in the query.If the query contains more than one FROM clause (subqueries, derived tables, UNION), the optimizer may select several indexed views to manage a query with multiple FROM clauses. In exceptional situations, the optimizer may collapse two FROM clauses into one (subquery to join or derived table to join transformation). If that happens, the indexed view substitution may cover more than one FROM clause in the original query.

Example queries demonstrating these conditions are presented at the end of this chapter. Allowing the query optimizer to determine which indexes, if any, to use in the query execution plan is the recommended practice.

The NOEXPAND option forces the query optimizer to treat the view like an ordinary table with a clustered index. In this case, the indexed view must be referenced directly in the FROM clause. For example,

SELECT Column1, Column2, ... FROM Table1, View1 WITH (NOEXPAND)WHERE ...

Alternatively, the user can explicitly exclude indexed views from consideration by using the EXPAND VIEWS option at the end of the query. For example,

SELECT Column1, Column2, ... FROM Table1, View1 WHERE...OPTION (EXPAND VIEWS)

When this option is used, the query optimizer ignores all view indexes when estimating the low-cost method of covering the columns referenced in the query.


Design Considerations

Identifying an appropriate set of indexes for a database system can be complex. While there are numerous possibilities to consider when designing ordinary indexes, adding indexed views to the schema dramatically increases the complexity of the design and the potential results. Indexed views can be used on any subset of tables referenced in the query, any subset of the conditions in the query for that subset of tables, grouping columns, and aggregate functions (for example, SUM).

Indexes on tables and indexed views should be designed concurrently to obtain the best results from each construct. Because both indexes and indexed views may be useful for a given query, designing them separately can lead to redundant recommendations that incur high storage and maintenance overhead. While tuning the physical design of a database, trade-offs must be made between the performance requirements of a diverse set of queries and updates that the database system must support. Therefore, identifying an appropriate physical design for indexed views is a challenging task, and the Index Tuning Wizard should be used wherever possible.

Query optimization cost can increase substantially if there are many indexed views that the query optimizer may consider for a particular query. A query optimizer may consider all indexed views that are defined on any subset of tables in the query. Each view has to be parsed and then investigated for the potential substitution before it is rejected. This may take same time, especially if there are hundreds of such views for a given query.

A view must meet several requirements before you can create a unique clustered index on it. During the design phase, consider these requirements:

  • The view, and all tables referenced in the view, must be in the same database and have the same owner.

  • The indexed view does not need to contain all the tables referenced in the query to be used by the optimizer.

  • A unique clustered index must be created before any other indexes can be created on the view.

  • Certain SET options (discussed later in this document) must be set correctly when the base tables, view, and index are created, and whenever data in the base tables and view are modified. In addition, the query optimizer will not consider the indexed view unless these SET options are correct.

  • The view must be created using schema binding and any user-defined functions referenced in the view must also be created with the SCHEMABINDING option.

  • Additional disk space will be required to hold the data defined by the indexed view.

Consider these guidelines when designing indexed views:

  1. Design indexed views that can be used by several queries or multiple operations. For example, an indexed view that contains the SUM of a column and the COUNT_BIG of a column can be used by queries that contain the functions SUM, COUNT, COUNT_BIG, or AVG. The queries will be faster because only a small number of rows from the view need to be retrieved rather than the full number of rows from the base tables and a portion of the computations required for performing the AVG function have already been done.

  2. Keep the index compact. By using the fewest possible columns and bytes, the optimizer gains maximum efficiency in locating the row data. Conversely, if a large clustered index key is defined, any secondary, nonclustered indexes defined on the view will be significantly larger because the nonclustered index entries will contain the clustering key in addition to the columns defined by the index.

  3. Consider the size of the resulting indexed view. In the case of pure aggregation, the indexed view may not provide any significant performance gains if its size is similar to the size of the original table.

  4. Design multiple smaller indexed views that accelerate parts of the process. You may not be able to always design an indexed view that addresses the entire query. Consider creating several indexed views each performing a portion of the query.

Examples

A frequently executed query aggregates data in one database, aggregates data in another database, and then joins the results. Because an indexed view cannot reference tables from more than one database, you cannot design a single view to perform the entire process. However, you can create an indexed view in each database that does the aggregation for that database. If the optimizer can match the indexed views against existing queries, at least the aggregation processing will be faster, without the need to recode existing queries. Although the join processing is not faster, the overall query is faster because it uses the aggregations stored in the indexed views.

A frequently executed query aggregates data from several tables, and then uses UNION to combine the results. UNION is not allowed in an indexed view. You can design views to perform each of the individual aggregation operations. The optimizer can then select the indexed views to speed up queries with no need to recode the queries. While the UNION processing is not improved, the individual aggregation processes are improved.


Creating an Indexed View

The steps required to create an indexed view are critical to the successful implementation of the view.

  1. Verify that the SET options are correct for all existing tables that will be referenced in the view.

  2. Verify that your session's SET options are set correctly before creating any new tables and the view.

  3. Verify that the view definition is deterministic.

  4. Create the view using the WITH SCHEMABINDING option.

  5. Create the unique clustered index on the view.

Using SET Options to Obtain Consistent Results

Evaluating the same expression can produce different results in SQL Server if different SET options are active when the query is executed. For example, after the SET option CONCAT_NULL_YIELDS_NULL is set to ON, the expression 'abc' + NULL returns the value NULL. But after CONCAT_NULL_YIELDS_NULL is set to OFF, the same expression produces 'abc'. Indexed views require fixed values for several SET options to ensure that the views can be maintained correctly and return consistent results.

The SET options Table 10.1 must be set to the values shown in the Required Value column whenever the following conditions occur:

  1. The indexed view is created.

  2. There is an INSERT, UPDATE, or DELETE operation performed on any table participating in the indexed view.

  3. The indexed view is used by the query optimizer to produce the query plan.

If you are using an OLE DB or ODBC server connection, the only value that must be modified is the ARITHABORT setting. All DB LIB values must be set correctly either at the server level using sp_configure or from the application using the SET command. For more information about SET options, see "Using Options in SQL Server" in SQL Server Books Online for SQL Server 2000.

Using Deterministic Functions

The definition of an indexed view must be deterministic. A view is deterministic if all expressions in the SELECT list, as well as the WHERE and GROUP BY clauses, are deterministic. Deterministic expressions always return the same result when they are evaluated with a specific set of input values. Only deterministic functions may participate in deterministic expressions. For example, the DATEADD function is deterministic because it always returns the same result for any given set of argument values for its three parameters. GETDATE is not deterministic because it is always invoked with the same argument, yet the value it returns changes each time it is executed. Even if an expression is deterministic, if it contains float expressions, the exact result may depend on the processor architecture or version of microcode. To ensure data integrity in SQL Server 2000, such expressions can participate only as non-key columns of indexed views. Deterministic expressions that do not contain float expressions are called precise. Only precise deterministic expressions may participate in key columns and WHERE or GROUP BY clauses of indexed views.

Table 10.1. SET options
SET options | Required value | Default server value | OLE DB and ODBC value | DB LIB value
ANSI_NULLS | ON | OFF | ON | OFF
ANSI_PADDING | ON | ON | ON | OFF
ANSI_WARNING | ON | OFF | ON | OFF
ARITHABORT | ON | OFF | OFF | OFF
CONCAT_NULL_YIELDS_NULL | ON | OFF | ON | OFF
NUMERIC_ROUNDABORT | OFF | OFF | OFF | OFF
QUOTED_IDENTIFIER | ON | OFF | ON | OFF

Use the COLUMNPROPERTY function and IsDeterministic property to determine whether a view column is deterministic. Use the COLUMNPROPERTY function and IsPrecise property to determine if a deterministic column in a view with schemabinding is precise. COLUMNPROPERTY returns 1 if TRUE, 0 if FALSE, and NULL for invalid input (the column is not deterministic). For example, SELECT COLUMNPROPERTY(Object_Id ('Vdiscount1'),'SumDiscountPrice','IsPrecise') returns 0 because the SumDiscountPrice column references float column Discount from the table Order Details. Alternatively, the column SumPrice in the same view is both deterministic and precise. (Note: The view that this SELECT statement is based on can be found in the example section as View 1.)

Additional Requirements

In addition to the requirements listed in the design guidelines, the Using Set Options to Obtain Consistent Results and the Using Deterministic Functions sections, the following requirements must be met.

Base Table Requirements

  • Base tables must have the correct SET options set at the time the table is created or it cannot be referenced by the view with schemabinding.

  • Tables must be referenced by two-part names (owner.tablename) in the view definition.

Function Requirements

  • User-defined functions must be created using the WITH SCHEMABINDING option.

  • User-defined functions must be referenced by two-part names (owner.function).

View Requirements

  • The view must be created using the WITH SCHEMABINDING option.

  • The view must reference only base tables in the same database, not other views.

Syntax Restrictions on the View Definition

COUNT( [image] ) | ROWSET function
Derived table | self-join
DISTINCT | STDEV, VARIANCE, AVG
Float [image] , text, ntext, image columns | Subquery
full-text predicates (CONTAIN, FREETEXT) | SUM on nullable expression
MIN, MAX | TOP
OUTER join | UNION

Note

The indexed view may contain float columns; however, such columns cannot be included in the clustered index key.

If GROUP BY is not used, expressions cannot be used in the SELECT list. If GROUP BY is present, the VIEW definition must contain COUNT_BIG( [image]) and must not contain HAVING, CUBE, or ROLLUP. These restrictions are applicable only to the indexed view definition. A query can use an indexed view in its execution plan even if it does not satisfy the GROUP BY restrictions.

Index Requirements

  • The user executing the CREATE INDEX statement must be the view owner.

  • If the view definition contains a GROUP BY clause, the key of the unique clustered index can reference only the columns specified in the GROUP BY clause.


Maintaining Indexed Views

SQL Server automatically maintains indexed views similar to any other index. In the case of ordinary indexes, each index is tied directly to a single table. With each INSERT, UPDATE, or DELETE operation performed on the underlying table, the index is updated accordingly so that the values stored in the index are always consistent with the table.

Indexed views are similarly maintained; however, if the view references several tables, updating any of them may require updating the indexed view. Unlike ordinary indexes, a single row insert into any of the participating tables may cause multiple row inserts into the indexed view. The same is true for updates and deletes. Consequently, maintaining of an indexed view may be more expensive than maintaining an index on the table.

In SQL Server 2000, some views can be updated. When a view is updatable, the underlying base tables are modified directly through the view using INSERT, UPDATE, and DELETE statements. Creating an index on a view does not prevent the view from being updatable.


Summary

In this chapter we covered indexed views: the requirements for creating indexed views and how they can be beneficial for your applications.


Chapter 11. Trigger Alternatives

Traditionally, SQL Server has always had AFTER triggers. Oracle has a concept of INSTEAD OF triggers and with providing this functionality in SQL Server 2000, Microsoft has again tried to bridge in the gap between SQL Server and Oracle. With the introduction of these triggers, all views are now updateable in SQL Server 2000.


Types of Triggers

AFTER Triggers

After triggers are executed after the action of the INSERT, UPDATE, or DELETE statement is performed. Specifying AFTER is the same as specifying FOR, which is the only option available in earlier versions of SQL Server. AFTER triggers can be specified only on tables.

INSTEAD OF Triggers

INSTEAD OF triggers are executed in place of the usual triggering action. INSTEAD OF triggers can also be defined on views with one or more base tables, where they can extend the types of updates a view can support.

INSTEAD OF triggers give you the ability to evaluate the changes that would have taken place if the data modification statement had actually executed. Like AFTER triggers, each INSTEAD OF trigger gives you access to two virtual tables called Inserted and Deleted. For a DELETE trigger, the virtual table Deleted holds all the deleted rows, and for an INSERT trigger, the virtual table Inserted holds all the new rows. (For INSTEAD OF triggers, the Deleted table holds the rows that would have been deleted and the Inserted table holds the rows that would have been inserted had the modification taken place.) An UPDATE trigger populates both the Inserted and Deleted tables; the Deleted table stores the old version of the rows, and the Inserted table stores the new version.

With all triggers, the Inserted and Deleted tables change their structure to reflect the columns and data types of the table the trigger is attached to. For example, if you have a trigger on the Titles table in the Pubs database, the Inserted and Deleted tables will have the same columns as the Titles table with the same names and data types.

Table 11.1 compares an INSTEAD OF trigger with an AFTER trigger. Following is a simple example. Suppose you want to make sure that when someone updates any prices in the Titles table in the Pubs database, the change is no more than 10 percent of the original price. You need to look at both the old price value and the new value to see whether the change is more than 10 percent. You can get the old value from the Deleted table and the new value from Inserted. You must also make sure that the primary key title_id doesn't change so that you can use that key to relate rows in Deleted to the corresponding rows in Inserted.

T-SQL Code 1 contains the AFTER trigger that performs the price check. In SQL Server 2000, you can use the word AFTER instead of FOR; the meaning is equivalent. If your application includes both kinds of triggers, you might use the word AFTER for consistency and to clearly distinguish the two kinds of triggers. And triggers you created with the word FOR in previous SQL Server releases will still work. Because the AFTER trigger fires after a client updates the Titles table, the trigger must force a rollback if it encounters an error. If no ROLLBACK TRANSACTION occurs, SQL Server has accepted the change and lets it persist.

Table 11.1. Comparison of AFTER and INSTEAD OF triggers
Function | AFTER trigger | INSTEAD OF trigger
Applicability | Tables | Tables and views
Quantity per table or view | Multiple per triggering action (UPDATE, DELETE, and INSERT) | One per triggering action (UPDATE, DELETE, and INSERT)
Cascading references | No restrictions apply | Not allowed on tables that are targets of cascaded referential integrity constraints.
Execution

After:

Constraint processing

Declarative referential actions

Inserted and deleted tables creation

Triggering action

Before:

Constraint processing

In place of:

After:

The triggering action inserted and deleted tables creation
Order of execution | First and last execution may be specified | Not applicable
text, ntext, and image column references in inserted and deleted tables | Not allowed | Allowed

CREATE TRIGGER AFTER_UPDATE_TITLES
ON titles
FOR update
AS
-- We will assume that more than one row could have been updated.
-- The inserted and deleted tables will have to be joined on their
-- primary key, so we can't allow changes to the primary key.
IF @@rowcount = 0 RETURN
IF UPDATE(title_id) BEGIN
  PRINT 'updates to primary key title_id are not allowed'
  ROLLBACK TRAN
  RETURN
END

IF UPDATE (price)
   IF (SELECT MAX(ABS((i.price - d.price)*100/d.price))
     FROM inserted i JOIN deleted d
        ON i.title_id = d.title_id)  > 10
  BEGIN
     PRINT 'Price Change not allowed'
     ROLLBACK TRAN
     RETURN
  END
GO

Now look at the INSTEAD OF trigger that T-SQL Code 2 shows. This trigger fires before the changes happen, but the rows that would have been affected are available in the Inserted and Deleted tables. The basic comparison for price changes greater than 10 percent is the same in this trigger as in the AFTER trigger.

CREATE TRIGGER INSTEAD_OF_UPDATE_TITLES
ON titles
INSTEAD OF update
AS

IF @@rowcount = 0 RETURN
IF UPDATE(title_id) BEGIN
  PRINT 'updates to primary key title_id are not allowed'
  RETURN
END

IF UPDATE (price)
  BEGIN
   IF (SELECT MAX(ABS((i.price - d.price)*100/d.price))
     FROM inserted i JOIN deleted d
        ON i.title_id = d.title_id)  > 10
      BEGIN
     PRINT 'Too big of a price change'
     RETURN
      END

-- Because the update passed the tests, we have to
-- actually do the update; this section updates only
-- the price field; if other columns also changed,
-- we would have to account for them.

    UPDATE t
     SET price = i.price
     FROM titles t join inserted i
          ON i.title_id = t.title_id
    WHERE t.title_id = i.title_id
  END
RETURN

The most difficult aspect of using the INSTEAD OF trigger is determining what to have SQL Server do if it deems the update acceptable. If you want to carry out the intended update, you could delete all the rows that match the rows in the Deleted table and insert all the rows from the Inserted table. But in this case, the Titles table has foreign-key references from other tables, so you can't delete rows from Titles. My simple solution, which the T-SQL Code 2 shows, works on the premise that the price was updated, so the trigger can update the price values in the Titles table based on the Inserted table's contents. If the original UPDATE had set many columns to new values, this trigger would be more difficult to write.

Both AFTER and INSTEAD OF triggers prevent updates to the primary key of the Titles table, and both triggers prevent updates to the price column that would change the price by more than 10 percent. Of course, you probably wouldn't have both these triggers on the Titles table; you would choose to have one or the other because you need to validate the price change only once.

Core Difference Between INSTEAD OF and AFTER Triggers

Besides using the words INSTEAD OF in place of FOR or AFTER, INSTEAD OF triggers behave differently from AFTER triggers. For example, you can have only one INSTEAD OF trigger on each table for each action (INSERT, UPDATE, and DELETE), and you can't set a firing order for INSTEAD OF triggers. (In SQL Server 2000, you can specify which AFTER trigger should execute first and which should execute last.)

Also, you can't combine INSTEAD OF triggers and foreign keys with CASCADE on a table. For example, if the Sales table has a foreign-key constraint that references the Titles table and specifies CASCADE as the response to DELETE operations on Titles, you'll get an error message if you try to create an INSTEAD OF trigger for DELETE on Sales. However, you can have INSTEAD OF triggers for INSERT or UPDATE. Similarly, if you already have an INSTEAD OF trigger on Sales, you can't alter the table to add a foreign-key constraint with the CASCADE action for the same data-modification operation.

Another difference is that INSTEAD OF triggers can never be recursive, regardless of the setting of the Recursive Triggers database option. For example, if you execute an INSTEAD OF trigger for INSERT into Titles and the trigger performs an INSERT into Titles, the second INSERT won't invoke the INSTEAD OF trigger. Instead, SQL Server will process the INSERT as if no INSTEAD OF trigger existed for INSERT, and any constraints and AFTER triggers will take effect.

Although you might think INSTEAD OF triggers and AFTER triggers could be interchangeable in some situations, the purpose and the real power of INSTEAD OF triggers is to let you update a certain class of views that aren't usually updatable. In fact, you can't create an AFTER trigger on a view, but you can create an INSTEAD OF trigger. For example, you usually can't execute DELETE operations on a view that is based on a join. However, you can write an INSTEAD OF DELETE trigger. The trigger has access (through the Deleted table) to the rows of the view that would have been deleted had the view been a real table. Similarly, in an INSTEAD OF UPDATE or INSTEAD OF INSERT trigger, you can access the new rows through the Inserted table.

Which Trigger to Pick?

The AFTER trigger might appear less efficient because it sometimes needs to undo work that has already been done. If you think your table will have numerous violations that your trigger will need to correct, you might want to choose the INSTEAD OF trigger. However, if the vast majority of your updates will be acceptable, the INSTEAD OF trigger will have more work to do and thus be less efficient. In the case of AFTER triggers, the contents of the Inserted and Deleted tables are directly available from the transaction log. By the time the AFTER trigger executes, the data modification has happened, SQL Server has logged the changes, and the changed records are available internally to SQL Server. In fact, when you use AFTER triggers, you can think of the Inserted and Deleted tables as views of the transaction log.

When INSTEAD OF triggers fire, SQL Server hasn't yet made any changes and consequently hasn't logged any changes. SQL Server builds worktables to hold the inserted and deleted records as if the modification had occurred. Then, if the modification takes place, SQL Server must make the changes and log them, adding to the work of creating the worktables. Besides the extra work, another reason INSTEAD OF triggers are the less desirable type has to do with updates, such as those in the price-changing example. If many of the table's columns could be included in the SET clause in the UPDATE statement, the trigger would be cumbersome to write, and each column would have to be checked to see whether it had a different value in the Deleted or Inserted tables.


Designing INSTEAD OF Triggers

The primary advantage of INSTEAD OF triggers is that they allow views that would not be updatable to support updates. A view comprising multiple base tables must use an INSTEAD OF trigger to support inserts, updates, and deletes that reference data in the tables. Another advantage of INSTEAD OF triggers is that they allow you to code logic that can reject parts of a batch while allowing other parts of a batch succeed.

An INSTEAD OF trigger can take actions such as ignoring parts of a batch, not processing a part of a batch and logging the problem rows, and taking an alternative action if an error condition is encountered. (Note: INSTEAD OF DELETE and INSTEAD OF UPDATE triggers cannot be defined on a table that has a foreign key defined with a DELETE or UPDATE action.) Coding logic as part of an INSTEAD OF trigger prevents all applications accessing the data from having to reimplement the logic.

In the following sequence of Transact-SQL statements, an INSTEAD OF trigger updates two base tables from a view. In addition, two approaches to handling errors are shown. (1) Duplicate inserts to the Person table are ignored, and the information from the insert is logged in the PersonDuplicates table. (2) Inserts of duplicates to the EmployeeTable are turned into an UPDATE statement that retrieves the current information into the EmployeeTable without generating a duplicate key violation.

The Transact-SQL statements create two base tables, a view, a table to record errors, and the INSTEAD OF trigger on the view. These tables separate personal and business data and are the base tables for the view.

CREATE TABLE Person
   (
    SSN         char(11) PRIMARY KEY,
    Name        nvarchar(100),
    Address     nvarchar(100),
    Birthdate   datetime
   )
GO
CREATE TABLE EmployeeTable
   (
    EmployeeID       int PRIMARY KEY,
    SSN              char(11) UNIQUE,
    Department       nvarchar(10),
    Salary           money,
    CONSTRAINT FKEmpPer FOREIGN KEY (SSN)
    REFERENCES Person (SSN)
   )

This view reports all relevant data from the two tables for a person:

CREATE VIEW Employee AS
SELECT P.SSN as SSN, Name, Address,
       Birthdate, EmployeeID, Department, Salary
FROM Person P, EmployeeTable E
WHERE P.SSN = E.SSN

You can record attempts to insert rows with duplicate social security numbers. The PersonDuplicates table logs the inserted values, the name of the user who attempted the insert, and the time of the insert.

CREATE TABLE PersonDuplicates
   (
    SSN           char(11),
    Name          nvarchar(100),
    Address       nvarchar(100),
    Birthdate     datetime,
    InsertSNAME   nchar(100),
    WhenInserted  datetime
   )

The INSTEAD OF trigger inserts rows into multiple base tables from a single view. Attempts to insert rows with duplicate social security numbers are recorded in the PersonDuplicates table. Duplicate rows in the Employee Table are changed to update statements.

CREATE TRIGGER IO_Trig_INS_Employee ON Employee
INSTEAD OF INSERT
AS
BEGIN
SET NOCOUNT ON
-- Check for duplicate Person. If no duplicate, do an insert.
IF (NOT EXISTS (SELECT P.SSN
      FROM Person P, inserted I
      WHERE P.SSN = I.SSN))
   INSERT INTO Person
      SELECT SSN,Name,Address,Birthdate,Comment
      FROM inserted
ELSE
-- Log attempt to insert duplicate Person row in PersonDuplicates table.
   INSERT INTO PersonDuplicates
      SELECT SSN,Name,Address,Birthdate,SUSER_SNAME(),GETDATE()
      FROM inserted
-- Check for duplicate Employee. If no duplicate, do an insert.
IF (NOT EXISTS (SELECT E.SSN
      FROM EmployeeTable E, inserted
      WHERE E.SSN = inserted.SSN))
   INSERT INTO EmployeeTable
      SELECT EmployeeID,SSN, Department, Salary,Comment
      FROM inserted
ELSE
-- If duplicate, change to UPDATE so that there will not
-- be a duplicate key violation error.
   UPDATE EmployeeTable
      SET EmployeeID = I.EmployeeID,
          Department = I.Department,
          Salary = I.Salary,
          Comment = I.Comment
   FROM EmployeeTable E, inserted I
   WHERE E.SSN = I.SSN
END


Summary

In this chapter we covered the topic of indexed views and the advantages of using them and the differences between the indexed views and the traditional AFTER views.


Chapter 12. Meta Data Services Enhancements

SQL Server 2000 Meta Data Services extends and renames the former component known as Microsoft Repository. Meta Data Services extends repository technology by introducing a new browser for viewing data in a repository database, new Extensible Markup Language (XML) interchange support, and new repository engine features.

SQL Server 2000 Meta Data Services is an object-oriented repository technology that can be integrated with enterprise information systems or with applications that process meta data. A number of Microsoft technologies use Meta Data Services as a native store for object definitions or as a platform for deploying meta data. One way in which SQL Server 2000 uses Meta Data Services is to store versioned DTS packages. In Microsoft Visual Studio, Meta Data Services supports the exchange of model data with other development tools.

You can use Meta Data Services for your own purposes as a component of an integrated information system, as a native store for custom applications that process meta data, or as a storage and management service for sharing reusable models. You can also extend Meta Data Services to provide support for new tools for resale or customize it to satisfy internal tool requirements.


Meta Data Fundamentals

Microsoft SQL Server 2000 Meta Data Services provides a way to store and manage meta data about information systems and applications. This technology serves as a hub for data and component definitions, development and deployment models, reusable software components, and data warehousing descriptions.

Product components consist of the repository engine, tools, APIs, standard information models, a browser, and a Software Development Kit (SDK). Integrating a meta data management service into a product, a development environment, or an information system requires understanding what meta data is and how it is used. The topics in this section describe fundamental concepts and usage strategies that identify ways in which meta data management technology is applied.

Microsoft SQL Server 2000 Meta Data Services is a set of services that allows you to manage meta data. Using Microsoft Meta Data Services requires understanding meta data characteristics. If you are new to the concept of meta data, this overview will help you learn about the type of data that Meta Data Services manages.

Meta data describes the structure and meaning of data, as well as the structure and meaning of applications and processes. It is important to remember that meta data is abstract, has a context, and can be used for multiple purposes in a development environment.

Meta Data Is Abstract

The simple act of describing real-world phenomena generates abstract information that qualifies as meta data. For example, describing natural phenomena such as rain, wind, and sunshine requires abstractions like the concept of weather. Weather can be further abstracted by defining concepts such as temperature, precipitation, and humidity. In data design, real-world phenomena are also described in abstract terms. People, places, things, and numbers are grouped or designated as employee, customer, or product data. In software design, the application and database structures that represent or store data can be abstracted into meta data classification schemes that make sense to developers and designers. A table or form is derived from an object, which, in turn, can be derived from a class.

There are multiple levels of abstraction in meta data. You can describe a data instance, then describe that description, and continue to describe subsequent descriptions until you reach some practical limit. Typically, meta data descriptions used in software development extend to two or three levels of abstraction. In real terms, a data instance of "loan table" can be described as a database table name. A database table can be described as a database table object. Finally, a database table object can be described by an abstract class that formalizes the fixed set of characteristics to which all derived objects must conform.

Meta Data Has Context

The distinction between data and meta data is often called the type/instance distinction. Model designers articulate types (such as classes or relationships) and software developers articulate instances (such as a Table class or a "table has columns" relationship).

The distinction between instance and type is context sensitive. What is meta data in one scenario becomes data in another scenario. For example, in a typical relational DBMS, the system catalog describes the tables and columns that contain the data. You can think of the data in the system catalog as meta data because it describes data definitions. However, with the right software tool, you can manipulate it as you would manipulate any other data. Examples of manipulating meta data include viewing data lineage or table versioning information, or identifying all tables that express financial data by searching for columns that have a currency-based data type. In this scenario, standard meta data like the system catalog becomes data you can manipulate.

Meta Data Has Multiple Purposes

You can work with meta data type and instance information just as you would with any kind of application or data design elements. Expressing design information as meta data, especially standard meta data, opens up new possibilities for reuse, sharing, and multiple tool support.

For example, defining data objects as meta data enables you to see how they are constructed and versioned. Versioning support provides a way to view, branch, or retrieve any historical version of a particular DTS package or data warehousing definition. When you develop code based on meta data, you can define a structure once and then reuse it to create multiple instances that can be versioned for specific tools and applications. You can also create new relationships among existing meta data types to support a new application design.

Meta Data Management

Tool developers can make use of Microsoft SQL Server 2000 Meta Data Services by adding meta data management support in products they provide. Meta Data Services provides a platform for building meta data management capability into dedicated tools or into add-on features of existing tools.

Shared meta data is a way to deploy data and application structures across heterogeneous platforms and development environments. It provides common definition so that tools and applications can interpret the same meta data definition and transform it into application-specific structures. Meta data is an integration point because it is abstract, containing essential details that remain constant regardless of the implementation strategy. This flexibility makes it ideally suited for design purposes because it allows you to separate design from implementation. When you work with predefined meta data, you can implement a specific design using the development tool that best serves your needs.

As you incorporate meta data into your information systems, you need tools that can keep track of the meta data you create, how it is used, and how it can be reused in subsequent projects.

Meta Data Services helps you manage meta data by providing a platform that enables you to do the following:

  • Store meta data constructs.

  • Version meta data objects and relationships so that you can work with current and historical editions with equal ease.

  • Allocate workspaces to isolate modifications to a specific set of objects.

  • Import and export meta data structures in Extensible Markup Language (XML) format to work with your meta data in a variety of environments.


Information Model Fundamentals

An information model is a set of meta data types that describe a tool, application, data structure, or information system. You can model a business process, for example, to describe the progression of an order as it moves from order entry to final invoicing. If you model a database application, your information model describes the tables and columns that are supported by the application. If your goal is to define an application for booksellers, your information model will include elements that describe books, authors, and publishers, the kinds of data a bookseller application would need to manipulate.

Notice that these examples depict types of data rather than instances of data. The first example describes an order process, not the specific orders placed by a customer. Similarly, an information model for a database application describes tables, keys, constraints, and stored procedures but not the actual data these elements store and manipulate. In the same way, the information model for the bookseller application describes the concept of a book but not data about individual books. As you can see, information models articulate things that are always two steps removed from user instance data.

Information Model Building Blocks

Information models are described by classes, relationships, and properties. A class is a template that defines the characteristics of objects. A class represents entities in an information model. In previous examples, an order, a table, a key, a constraint, and a stored procedure represent different classes.

A relationship type defines a template to which stored relationships must conform. For example, you can define the type of relationship between a table and a column, a column and a data type, or a schema and a table. A relationship type defines a set of criteria that describe how two objects relate.

A property is a template to which stored property values must conform. For example, when you store an invoice total, you must store a currency value.

Standard Information Models

Microsoft SQL Server 2000 Meta Data Services distributes an implementation of the open information model (OIM) that provides standard meta data types that tools can use. The OIM is a generic set of information models that describe object modeling, database modeling, and component reuse.

You can extend the OIM by adding custom elements. For example, if you are creating an application that requires elements that are not included in the OIM, you can add those elements to complete your design. Although the OIM is not required, deploying an OIM-based strategy provides integration possibilities that are not otherwise achievable.

Importance of Information Models

In SQL Server 2000 and in other Microsoft products that integrate with Meta Data Services, OIM-based models are predefined, installed, and operational. No action is required on your part if all you want to do is use these services. However, if you want to build your own tools and applications that work with meta data types or if you want to create or extend an information model, you will need to know all about information models. Information models are the key to integrating Meta Data Services with other tools and technologies.


New Features in Meta Data Services

Meta Data Browser Enhancement

Meta Data Browser is a new tool in SQL Server 2000 Meta Data Services that you can use to browse the contents of a repository database. You can run Meta Data Browser when you select Meta Data Services. Meta Data Services is available for each copy of SQL Server you install.

XML Encoding Enhancements

SQL Server 2000 Meta Data Services now uses Extensible Markup Language (XML) encoding in native mode. You can import, export, and publish repository meta data in a format that more closely matches your information model. XML encoding supercedes the XML Interchange Format (XIF) that was part of previous versions of the software.

XML in Meta Data Services

SQL Server 2000 Meta Data Services supports Extensible Markup Language (XML) Encoding of information models for the purpose of importing, exporting, and publishing meta data in XML. You can exchange meta data between two repository databases, between a repository database and an application, or between two applications that can interpret the same XML format.

Meta Data Services encodes, exchanges, and decodes XML documents for you. This functionality is provided through dual interfaces so that you can manage these operations from code.

The XML format supported by Meta Data Services is defined by the Meta Data Coalition (MDC) open information model (OIM) XML encoding format. This format defines rules for generating XML that is based on an information model. Applying these rules enables Meta Data Services to generate XML that corresponds to your information model. These same rules also enable Meta Data Services to convert an XML document back into repository instance data.

XML encoding provided with this release of Meta Data Services supersedes the XML Interchange Format (XIF) that was part of previous versions of the software.


Summary

In this chapter we covered meta data services and the new feature sets added to it in SQL Server 2000.


Chapter 13. Tool Enhancements

There have been several enhancements in the graphical tools like Query Analyzer, Enterprise Manager, Profiler, and the wizards. We have already discussed the enhancements to SQL Server Profiler and the index tuning wizard in Chapter 6. In this chapter we will go over the enhancements and the new feature sets added to the Query Analyzer and Enterprise Manager to make administration easy.


Query Analyzer

SQL Query Analyzer now includes Object Browser, which allows you to navigate through the browser and get information (such as parameters and dependencies) about database objects, including user and system tables, views, stored procedures, extended stored procedures, and functions. The Object Browser also supports generating scripts to either execute or create objects. Other enhancements include server tracing and client statistics that show information about the server-side and client-side impact of a given query.

SQL Query Analyzer includes a stored procedure debugger. SQL Query Analyzer also includes templates that can be used as the starting points for creating objects such as databases, tables, views, and stored procedures.


Templates

Everyone uses templates in one form or another. Every time you copy and paste a line of code or a sentence and change the result slightly, you use a template to reduce your workload. Programmers type the same structures in a program over and over, and almost all C and C++ tools include templates that build a skeleton for any application, module, or function. For example, when you add a function to a Visual Basic (VB) form or module, the IDE automatically creates a function template. These days, most development environments generate some type of skeleton routine when you add a function.

Native SQL Server tools include features that let you create a form of template by reverse-engineering a script of the code you need to generate an object. However, when you use these native tools in new development, you must considerably edit the script you generate.

Using Templates in SQL Query Analyzer

Templates are boilerplate files containing SQL scripts that help you create objects in the database. SQL Server 2000 provides a variety of templates in the Templates\SQL Query Analyzer directory. Among the templates provided are those that create databases, tables, views, indexes, stored procedures, triggers, statistics, and functions. In addition, there are templates that help you to manage extended properties, linked servers, logins, roles, and users, and to declare and use cursors.

The template scripts provided with SQL Query Analyzer contain parameters to help you customize the code. Template parameter definitions use this format: <parameter_name, data_type, value> where parameter_name is the name of the parameter in the script, data_type is the data type of the parameter, and value is the value that is to replace every occurrence of the parameter in the script. Use the Replace Template Parameters dialog box to insert values into the script. (Note: You can use the Replace Template Parameters dialog box to specify values any time a parameter definition is used in code. For example, when you execute a function from Object Browser, the function that is written to the Editor pane contains parameter definitions for any arguments in the function. You can, therefore, use the Replace Template Parameters dialog box to specify argument values.)

To use a template, follow these steps.

  1. Open a Query window.

  2. Click Insert Template on the toolbar.

  3. In the Insert Template dialog box, specify the template to open. By default, template files have the extension .tql.

  4. When the template is displayed in the Editor pane, select Replace Template Parameters from the Edit menu.

  5. In the Replace Template Parameters dialog box, specify values for the parameters.

  6. To insert the specified values into the script in the Editor pane, click Replace All.

  7. Save the file under a different name.

Building Your Own Template

Let's build a template, then use it in a few sections of code to demonstrate its benefits. I use templates in two ways: to develop objects and to reduce the amount of typing I have to do when I write a section of code. The templates provide handy snippets of code that I use when I compile a series of objects on a server.

To build a template, first open Query Analyzer and enter the desired text in the design window. Second, after you enter the text, click File, Save from the menu. When the File dialog box appears, enter the template's name. Third, change SQL Query ( [image].sql) files in the Save as Type drop-down list to Template SQL ( [image].tql) files. Next, if you installed SQL Server on the C drive, open the C:\program files\microsoft sql server\80\tools\templates\sql Query Analyzer folder. (If SQL Server isn't installed on the C drive, replace C with the name of the drive on which you installed SQL Server.) This folder already contains folders that correspond to the default templates that are installed with the tool. Then, after you save your template and right-click to refresh the treeview in the left pane, your template appears. Finally, press Ctrl+N to open a new query window, then drag your new template onto the window. The outline of a stored procedure appears.

You can perform even more tricks with this amazing Query Analyzer template tool. You can change the parameter names for each project. Also, the type names you include are for programmer reference and don't affect the values you replace in the template. After you drop this template into the query window and press Shift+Ctrl+M, the Replace Template Parameters dialog box appears (Figure 13.1).

Figure 13.1. Replace Template Parameters dialog box

The dialog box lets you change the default values in the third column to suit the batch you're working on. Then you can type the new name for the procedure, press the down arrow twice, and type the current date into the dialog box. Finally, click Replace All.

Before I began building my templates, I examined many of my existing procedures and looked for repeated patterns or sections that I changed in every procedure. Then I set the repeating items as parameters in my templates. By using this technique, I shaved minutes off the development time for every stored procedure.


Copy Database Wizard

This is a new wizard in the Enterprise Manager. Users can run the Copy Database Wizard to upgrade SQL Server version 7.0 databases to SQL Server 2000 databases. It can also be used to copy complete databases between instances of SQL Server 2000. The DTS package called Refresh_DATA shown in Chapter 27 utilizes the CopyData task to copy the data from one database to the other. This task is what drives the Copy Database Wizard.

Uses of Copy Database Wizard

The Copy Database Wizard allows you to copy or move databases between servers. You can move and copy databases between different instances of SQL Server 2000, and you can upgrade databases from SQL Server version 7.0 to SQL Server 2000.

To upgrade a SQL Server 7.0 database to a SQL Server 2000 database; first expand a server group, and then expand a server. Right-click the server, point to All Tasks, and then click Copy Database Wizard. Complete the steps in the wizard. (Important: After upgrading databases from SQL Server 7.0, run sp_updatestats (update statistics) against the database on the destination server to ensure optimal performance of the copied database.)

You also can simplify the administration and maintenance of your databases by integrating multiple instances of SQL Server 7.0 or SQL Server 2000 into a single instance of SQL Server 2000 or into several named instances on a single computer.

In addition to copying the actual databases, you also can copy associated meta data (for example, logins and objects from the master database required by the databases being copied).

Note

It is not possible to copy or move the model, msdb, or master databases using the Copy Database Wizard.

Copy and Move Process

To use the Copy Database Wizard, you must be a system administrator or a member of the sysadmin role. If you are running Microsoft Windows NT 4.0 or Microsoft Windows 2000, your user account must have administrator privileges.

To use the Copy Database Wizard, you must specify the following:

  • Source server on which the databases to be copied reside.

  • Destination server to which the databases are to be copied or moved. On the destination server, databases keep the same name they had on the source server. Therefore, only databases with source names that do not exist on the destination server can be moved or copied. Databases cannot be renamed during a move or copy operation. Name conflicts must be resolved manually prior to moving or copying databases.

  • Databases to be moved or copied. To avoid data loss or inconsistency, you must be sure there are no active sessions attached to the database being moved or copied. If there are active sessions, the Copy Database Wizard will not run.

  • Other objects to be copied or moved (for example, logins, shared objects from the master database, jobs and maintenance plans, and user-defined error messages).

  • Schedule for the copy or move operation, if you want it to run at a later time.

When moving databases between different servers or disk drives, the Copy Database Wizard copies the database to the destination server and verifies that it is online. The wizard does not delete the database from the source server. You must delete the files from the source server manually after you have verified that the copy is fully operational. When moving databases between two instances on the same server, the file system move operation is performed.


Summary

In this chapter we covered the Query Analyzer and the Copy Database Wizard feature sets that are new to SQL Server 2000.


Chapter 14. Backup and Recovery Enhancements

We discussed some of the backup and recovery enhancements in Chapter 4. As mentioned there, the major change in backup and recovery in SQL Server 2000 is the inclusion of "recovery models." In this chapter we delve a bit deeper into these features. SQL Server 2000 introduces a new, more easily understood model for specifying backup and restore options. The new model makes it clearer that increased or decreased exposure to losing work is being balanced against the performance and log space requirements of different plans. SQL Server 2000 introduces support for recovery to specific points of work using named log marks in the transaction log and the ability to do partial database restores.

To choose the appropriate backup and restore strategy, you must identify the requirements for the availability of your data. Your overall backup strategy defines the type and frequency of backups and the nature and speed of the hardware required for them.

It is strongly recommended that you test your backup and recovery procedures thoroughly. Testing helps to ensure that you have the required backups to recover from various failures and that your procedures can be executed smoothly and quickly when a real failure occurs.


Backup and Restore Architecture

The backup and restore components of Microsoft SQL Server 2000 allow you to create a copy of a database. This copy is stored in a location protected from the potential failures of the server running the instance of SQL Server. If the server running the instance of SQL Server fails, or if the database is somehow damaged, the backup copy can be used to recreate, or restore, the database.

Capabilities

SQL Server 2000 provides the following sophisticated backup and restore capabilities.

Options for how a database is backed up and restored

(1) A full database backup is a full copy of the database; (2) a transaction log backup copies only the transaction log; (3) a differential backup copies only the database pages modified after the last full database backup; (4) a file or filegroup restore allows the recovery of just the portion of a database that was on the failed disk. These options allow backup and restore processes to be tailored to how critical the data in the database is. Noncritical databases that can be easily recreated from some other source may have no backups, other databases may have simple backups that can recreate the database to the night before a failure, and critical databases may have sophisticated backups that will restore the database right up to the point of failure.

Control with the BACKUP and RESTORE statements

Users can execute the BACKUP and RESTORE statements directly from applications, Transact-SQL scripts, stored procedures, and triggers. It is more common, however, to use SQL Server Enterprise Manager to define a backup schedule and then let SQL Server Agent run the backups automatically according to the schedule. The Database Maintenance Plan Wizard can be used to define and schedule a full set of backups for each database. This fully automates the backup process, requiring minimal or no operator action.

Maintenance of a set of backup history tables in the MSDB database

The backup history tables record the backups for each database. If a database has to be restored, the Restore Database dialog box in the SQL Server Enterprise Manager presents the user with a list of all the backups available for the database. The Restore Database dialog box also has logic to display which set of the backups in the history can be used to restore the database in the shortest possible time. When the dialog box is displayed, the backups needed to restore the database are checked. If a user knows that one of the backups is not available (for example, if a tape cartridge was damaged or lost), the user can deselect that backup, and the SQL Server Enterprise Manager calculates a new restore process. When the user agrees with the restore process, the SQL Server Enterprise Manager restores the database, prompting for tapes as needed.

Backups that can be performed while the database is in use, allowing backups to be made of systems that must run continuously.

The backup processing and internal data structures of SQL Server 2000 are structured so that backups maximize their rate of data transfer with minimal effect on transaction throughput.

Fast data transfer rates for backup and restore operations, making SQL Server 2000 capable of supporting very large databases (VLDB)

The data structures in SQL Server 2000 databases and the backup and restore algorithms support high data transfer rates for backup and restore operations. SQL Server backup and restore operations can also run in parallel against multiple backup files or tape drives, which further increases the backup and restore data transfer rates.

RESTORE statement that recreates the database automatically if necessary

This eliminates the need to execute a separate CREATE DATABASE or CREATE DATABASE FOR LOAD statement if the database does not exist at the time the RESTORE statement is executed.

  • Interrupted backup and restore operations started near the point of the interruption when restarted.

  • Verification of an SQL Server 2000 backup before an attempt to restore the database. This includes verifying that the collation of the database is supported by the instance of SQL Server.

Backup and restore processes should be planned together. The administrators must first determine the criticality of the data in the database. They must determine if it is acceptable to just restore the database to a point such as the night before the failure, or if the database must be restored to a point as close as possible to the time of failure. They must also determine how long the database can be unavailable, whether it must be brought back online as quickly as possible, or if it does not need to be restored immediately.


Recovery Models

After the restore requirements are determined, the administrators can then plan a backup process that maintains a set of backups that will meet the restore requirements. The administrators can choose the backup processes that can be performed with the minimum effect on the system as it runs yet still meet the restore requirements. Based on the resource requirements, the administrators also choose the recovery model for the database. The recovery model balances logging overhead against the criticality of fully recovering the data. Following are the recovery models.

Simple Recovery

All data modifications made since the last backup are expendable or can be redone. This model has the lowest logging overhead, but it cannot recover past the end of the last backup.

With the simple recovery model, the database can be recovered to the point of the last backup. However, you cannot restore the database to the point of failure or to a specific time. To do that, choose either the full recovery or bulk-logged recovery model.

The backup strategy for simple recovery consists of (1) database backups and (2) differential backups (optional). To recover in the event of media failure, follow these steps.

  1. Restore the most recent full database backup.

  2. If differential backups exist, restore the most recent one.

Changes since the last database or differential backup are lost.

Full Recovery

The data is critical and must be recoverable to the point of failure. All data modifications are logged. All SQL Server 2000 recovery options are available.

The full recovery model uses database backups and transaction log backups to provide complete protection against media failure. If one or more data files is damaged, media recovery can restore all committed transactions. In-process transactions are rolled back.

Full Recovery provides the ability to recover the database to the point of failure or to a specific point in time. To guarantee this degree of recoverability, all operations, including bulk operations such as SELECT INTO, CREATE INDEX, and bulk loading data, are fully logged.

The backup strategy for full recovery consists of (1) database backups, (2) differential backups (optional), and (3) transaction log backups.

You can restore a database to the state it was in at the point of failure if the current transaction log file for the database is available and undamaged. To do so, follow these steps.

  1. Back up the currently active transaction log.

  2. Restore the most recent database backup without recovering the database.

  3. If differential backups exist, restore the most recent one.

  4. Restore each transaction log backup created since the database or differential backup in the same sequence in which they were created without recovering the database.

  5. Apply the most recent log backup (created in step 1), and recover the database.

Important: To protect against loss of transactions under the full recovery model, the transaction log must be protected against damage. It is strongly recommended that fault-tolerant disk storage be used for the transaction log.

Bulk-Logged Recovery

Certain bulk operations (bulk copy operations, SELECT INTO, text processing) can be replayed if necessary, so these operations are not fully logged. This model can recover only to the end of the last database or log backup.

The bulk-logged recovery model provides protection against media failure combined with the best performance and minimal log space usage for certain large-scale or bulk copy operations. The following operations are minimally logged:

  • SELECT INTO

  • Bulk load operations (bcp and BULK INSERT)

  • CREATE INDEX (including indexed views)

  • Text and image operations (WRITETEXT and UPDATETEXT)

Bulk-logged and full recovery are similar, and many users of the full recovery model use the bulk-logged model on occasion. In a bulk-logged recovery model, the data loss exposure for the bulk copy operations is greater than in the full recovery model. While the bulk copy operations are fully logged under the full recovery model, they are minimally logged and cannot be controlled on an operation-by-operation basis under the bulk-logged recovery model. Under the bulk-logged recovery model, a damaged data file can result in having to redo work manually.

In addition, the bulk-logged recovery model allows only the database to be recovered to the end of a transaction log backup when the log backup contains bulk changes. Point-in-time recovery is not supported.

Switching between full and bulk-logged recovery models is easy. It is not necessary to perform a full database backup after bulk copy operations are complete under the bulk-logged recovery model. Transaction log backups under this model capture both the log and the results of any bulk operations performed since the last backup.

The backup strategy for bulk-logged recovery consists of (1) database backups, (2) differential backups (optional), and (3) log backups.

Backing up a log that contains bulk-logged operations requires access to all data files in the database. If the data files are not accessible, the final transaction log cannot be backed up and all committed operations in that log are lost.

To recover in the event of media failure, follow these steps.

  1. Back up the currently active transaction log.

  2. Restore the most recent full database backup.

  3. If differential backups exist, restore the most recent one.

  4. Apply in sequence all transaction log backups created since the most recent differential or full database backup.

  5. Manually redo all changes since the most recent log backup.

Important: If the active transaction log is lost (for example, due to hardware failure on the disk containing the transaction log files), all transactions in that log are lost. To prevent loss of the active transaction log, place the transaction log files on mirrored disks.


Switching Recovery Models

You can switch a database from one recovery model to another in order to meet changing business needs. For example, a mission-critical online transaction-processing (OLTP) system requires full recoverability but periodically undergoes bulk load and indexing operations. The recovery model for the database can be changed to bulk-logged for the duration of the load and indexing operations and then returned to full recovery, thus increasing performance and reducing the required log space while maintaining server protection. (Note: Switching recovery models during a bulk load operation is permitted. The logging of the bulk operation changes appropriately.) Table 14.1 indicates the action to take when switching from one recovery model to another.

Table 14.1. Actions to be taken when you switch between recovery models
From | To | Action | Description
Full recovery | Bulk-logged | No action | Requires no change in backup strategy. Continue to perform periodic database, log, and (optionally) differential backups.
Full recovery | Simple recovery | Optionally back up the transaction log prior to the change. | Executing a log backup immediately before the change permits recovery to that point. After switching to the simple model, stop executing log backups.
Bulk-logged recovery | Full recovery | No action | Requires no change in backup strategy. Recovery to any point in time is enabled after the next log backup. If point-in-time recovery is important, execute a log backup immediately after switching.
Bulk-logged recovery | Simple recovery | Optionally back up the transaction log prior to the change. | Executing a log backup immediately before the change permits recovery to that point. After switching to the simple model, stop executing log backups.
Simple recovery | Full recovery | Back up the database after the change | Execute a database or differential after switching to the full recovery model. Begin executing periodic database, log, and (optionally) differential backups.
Simple recovery | Bulk-logged | Back up the database after the change. | Execute a database or differential backup after switching to the bulk-logged model. Begin executing periodic database, log, and (optionally) differential backups.


Summary

In this chapter we covered one of the most important topics in any RDBMS system: backup and recovery. With the concept of the recovery models, SQL Server 2000 provides much better control over devising a backup and recovery plan with optimal performance.


Chapter 15. Analysis Services Enhancements

What used to be called OLAP Services in SQL Server 7.0 is now called Analysis Services. Many new and improved features significantly enhance the analysis capabilities of the acclaimed OLAP Services introduced in SQL Server version 7.0. In SQL Server 2000, Analysis Services introduces data mining, which can be used to discover information in OLAP cubes and relational databases. One chapter on Analysis Services cannot do full justice to its feature sets, so at the end of the chapter I will list some suggestions for further reading in case you wish to delve more into this topic.


Cube Enhancements

SQL Server 2000 Analysis Services substantially extends the scalability and functionality of OLAP cubes. You can distribute cube data across multiple servers to provide more storage capacity, create linked cubes to distribute end-user access to information without duplicating cube data, create cubes that are updated in real time as data changes, and use a number of other new features to create cubes that address your specific business needs.

Distributed partitioned cubes

You can create distributed partitioned cubes by using remote partitions that distribute a cube's data among multiple analysis servers. A distributed partitioned cube is administered on a central analysis server.

Real-time OLAP

Real-time OLAP provides a multidimensional OLAP view of data that is continually updated as the underlying data changes. Real-time cubes implement real-time OLAP by using ROLAP storage for partitions and dimensions, new SQL Server 2000 indexed views for aggregations, and automatic notification by the SQL Server 2000 relational engine when data changes. Real-time cubes provide the capability to develop new categories of OLAP solutions such as call-center management, stock market analysis, or campaign management.

Linked cubes

A cube can be stored on a single analysis server and then defined as a linked cube on other analysis servers. Users connected to any of these analysis servers can then access the cube. This arrangement avoids the more costly alternative of storing and maintaining copies of a cube on multiple analysis servers. Linked cubes can be connected using TCP/IP or HTTP. To users, a linked cube looks like a regular cube.

Indexed views for aggregations

Indexed views for increased performance and flexibility are used instead of aggregation tables for ROLAP partitions if the partition's source data is stored in SQL Server 2000 and if certain criteria are met.

Cube processing

You can use lazy aggregations to make cube data available to users while aggregations are being calculated. When processing cubes for which the underlying data contains dimension key errors, you can elect to stop processing on key errors, stop processing after a specified number of errors, or ignore all key errors. Errors can be logged to a file for later review.

Calculated cells

You can specify formulas that apply to individual cells or to sets of cells in a cube. These formulas can contain conditional calculations that compute a new value for a cell or set of cells based on values in the cell or cells or on values in other cells in the cube. Calculated cells use multidimensional expressions (MDX), and you can specify calculations to be performed in multiple passes. Calculated cells can be used in complex financial modeling and budgeting applications; for example, you can specify a default value such as a percentage of a parent cell if the cell value is zero, or to use the actual value if it is not zero.

Drillthrough

Client applications that support drillthrough can now allow users to select a cube cell and retrieve a result set from the source data for that cell. You can use roles to control user access to the drillthrough functionality.

Actions

Actions enable users to act on the outcomes of their analyses. An action is a predefined operation that a user can initiate on a selected cube or portion of a cube. The operation can start an application with the selected item as a parameter or retrieve information about the selected item. A wizard is provided for help in creating actions.

Distinct count

You can use the new DistinctCount aggregate function to analyze the number of unique occurrences of events or transactions in your data, such as unique users visiting a Web site.

Hidden cube elements

You can hide complete cubes, dimensions, levels, measures, or member properties from end users who browse cubes with client applications. The visibility of these objects is controlled by the Visible property.

Named sets

You can create, name, and save sets of dimension members or set expressions in a cube. Client applications can use a named set like a dimension by placing the named set on an axis.

Default measures

Default measures can be specified for each cube and varied by role to control end users' default views of cubes.

Virtual cube editor

This new editor for virtual cubes is similar in function to Cube Editor. Virtual Cube Editor replaces and expands the functionality provided by the Calculated Member Manager Add-in for SQL Server version 7.0 OLAP Services.


Dimension Enhancements

This release of Microsoft SQL Server 2000 Analysis Services adds significant functionality to OLAP analysis with a number of new dimension types, features, and improvements.

Parent-child dimensions

A new parent-child dimension type supports hierarchies based on parent-child links between members in columns in a source table. Such hierarchies represent structures that include organization charts and part assemblies. Data members can be used to provide data for nonleaf members, such as the direct commission amount for a sales department manager or the individual salaries for all members in an organizational chart.

ROLAP dimensions

Extremely large dimensions can now be accommodated using the ROLAP storage mode. The dimension data remains in the database table and is not subject to the size limitations of MOLAP.

Write-enabled dimensions

The members of write-enabled dimensions can be updated through Analysis Manager and client applications that support dimension writeback. Roles are used to control dimension write access by client applications.

Changing dimensions

The new changing dimension type permits dimension members to be deleted, moved, added, or renamed without requiring the cube to be fully processed after changes. This increases the availability of cubes to client applications.

Dependent dimensions

Dependent dimensions permit improved aggregation design optimization by using knowledge of nonintersecting member combinations to reduce storage requirement estimation in the design algorithm.

Ragged dimensions

Ragged dimensions have at least one member whose logical parent is not in the level immediately above the member, such as is the case of countries that do not have a state or province level between the country and city levels.

Enhanced virtual dimensions

Virtual dimensions have better performance and greater flexibility than in earlier versions. They can be based directly on columns in another dimension's table and include multiple levels. Virtual dimensions are no longer limited to 760 members.

Custom rollup formulas and custom member formulas

Cube cell values associated with members can be calculated according to custom expressions rather than the aggregate functions of measures. The expressions can be applied to all the members in a level or individual members.

Automatic member grouping

System-generated member groups can be used to accommodate members that have more than 64,000 children. This feature can also be used to provide an intermediate level for drilldown between a level with few members and one with numerous members.

Default members

Default members can be specified for each dimension and varied by role. They control users' default views of cubes that include the dimension.

Dimension filters

A filter (WHERE clause expression) can be used to limit the dimension table rows that are included in the dimension. The filter is specified in the Source Table Filter property of the dimension.


Data Mining Enhancements

Data mining technology analyzes data in relational databases and OLAP cubes to discover information of interest. The data mining features of Microsoft SQL Server 2000 Analysis Services are incorporated in an open and extensible implementation of the new OLE DB for Data Mining specification. SQL Server 2000 includes data mining algorithms developed by Microsoft Research.

Relational and OLAP data mining

Analysis Services has incorporated data mining technology so that you can use it to discover information in relational databases and in OLAP cubes in Analysis Services. You can use the results of data mining to create a dimension that you can add to a cube to further analyze your data.

Microsoft decision trees

The Microsoft Decision Trees algorithm uses classification techniques to analyze data. It then constructs one or more decision trees that can be used to predict attributes or values for new data. For example, you can use this algorithm to analyze credit history data and predict the credit risk of new applicants.

Microsoft clustering

The Microsoft Clustering algorithm uses a nearest neighbor method to group records into clusters that share similar characteristics. Often, these characteristics may be hidden or nonintuitive.

Data mining user interface

Analysis Services provides new user interface wizards, dialog boxes, and editors to help you quickly perform data mining administrative tasks such as building data mining models and incorporating the results in OLAP cubes. You can browse a single decision tree model or the dependency network model of multiple trees produced when multiple attributes are predicted.

MDX extensions for data mining

Multidimensional Expressions (MDX) syntax has been extended to provide data mining capabilities in connection with OLAP cubes.

DTS tasks for data mining

The Analysis Services processing task has been enhanced to allow processing mining models, and a new data transformation services (DTS) task is provided that you can use to create predictions from mining models.

Data mining in client applications

Client applications for Analysis Services can use data mining algorithms to discover information in OLAP cubes by creating data mining models and virtual cubes.

Security enhancements

This release of Microsoft SQL Server 2000 Analysis Services includes features that provide more flexibility in controlling access to cube data, additional methods for authentication of users, and enhanced enforcement of roles.

Dimension security

You can use roles to control user access to dimensions. For each role, you can limit access to individual dimensions, levels, and members, and you can set various read and read/write permissions.

Enhanced cell security

Analysis Manager now includes dialog boxes that enable you to define cell security by setting role options to control user access to cube cells. You can limit a role's access to any combination of a cube's cells, and you can vary the read and read/write permission of the role.

Additional authentication methods

You can use HTTP authentication methods for client connections to analysis servers. Analysis Services also supports Windows 2000 negotiated protocol authentication.

Enhanced role enforcement

Role specifications are enforced on analysis servers installed on either file allocation table (FAT) or NTFS file systems.


Other Enhancements

This release of Microsoft SQL Server 2000 Analysis Services incorporates various enhancements to improve usability, processing performance, and MDX functionality.

Multiuser administration

Multiple users can administer an analysis server using Analysis Manager. Locking is applied only to the objects being edited and their dependent objects.

Integration of add-ins

The Archive and Restore Databases Add-in and the Copy and Paste Objects Add-in originally created for SQL Server version 7.0 OLAP Services are upgraded, installed, and fully integrated in the 2000 release. You can now use Analysis Manager to archive and restore SQL Server 2000 Analysis Services databases and to copy and paste objects in the Analysis Manager tree pane.

MDX enhancements

A number of new functions have been added to the Multidimensional Expressions (MDX) syntax that you can use for calculated members and increased analytical evaluation.

MDX builder

A new MDX Builder tool helps you create MDX expressions using drag-and-drop techniques. The MDX editing text box has been enhanced to provide color coding of keywords and functions, indication of parentheses balance, and smart tips that show function syntax.

Table aliases

In Cube Editor and Dimension Editor you can specify table aliases.

Active directory support

Analysis Services supports Active Directory, the directory service that is included with Microsoft Windows 2000. You can enable an analysis server so that it publishes Active Directory entries when the server starts and updates the entries when the server status changes.

Performance monitor counters

A number of performance counters are now available for the analysis server that can be used with Performance Monitor in Microsoft Windows NT 4.0 or System Monitor in Windows 2000.


Summary

In this chapter, we briefly covered the new feature sets of Analysis Services. It is a very powerful tool for building up data-warehousing applications. For further reading, you can refer to these books:

• Microsoft SQL Server 2000 Analysis Services Step by Step by Reed Jacobson. (Editor: OLAP Train, 2000)

• Microsoft OLAP Unleashed by Timothy Peterson (Sams, 2001)

• A Practical Guide to Microsoft OLAP Server® by John Shumate (Addison-Wesley, 2000)


Chapter 16. Distributed Partitioned Views

Microsoft SQL Server 2000 distributed partitioned views are a group of extensions to SQL Server features that support the transparent, horizontal partitioning of data across multiple database servers.

Although the database servers cooperate in managing the partitioned data, they operate autonomously. Each server is managed independently, has separate operational rules, and can support independent processes and data. These autonomous servers are known as federated servers. The world-record benchmark results published with the Microsoft Windows 2000 launch were achieved in a Windows 2000 clustered system using SQL Server 2000 federated database servers.

SQL Server 2000 is designed to support the processing growth requirements of large systems built with the Windows DNA architecture.


What Are Distributed Partitioned Views?

Distributed partitioned views allow a group of federated database servers running SQL Server 2000 to support the processing loads generated by a scaled-out, multitiered Windows 2000 cluster. Distributed partitioned views can horizontally partition data across federated database servers while providing the performance capacity of a load-balanced database cluster.

Horizontally partitioning a table means dividing a table into multiple smaller tables, called member tables. Each member table has the same format as the original table but only a portion (or partition) of the rows. Each table is placed on a separate resource (files or servers) to spread the processing load across the resources. For example, a company assigns customer IDs from 1 to 999. The Customers table may be partitioned into three member tables, with each member table having an equal customer ID range: Table1 has customers 1 through 333, Table2 has customers 334 through 666; and Table3 has customers 667 through 999.

If used without views, horizontal partitioning would force applications to implement logic to determine which member tables have the data requested by the user and dynamically build SQL statements referencing the tables. The application would require complex queries joining the member tables. Changing the way member tables are partitioned would also involve recoding the application. Views solve the problem by making the member tables look like one table. The SQL UNION operator combines result sets with identical formats into one. Because all the member tables have the same format, the result of SELECT [image] statements for each table have the same format, and can be unioned together to form a single result set that operates similar to the original table. The Customers example is shown in Figure 16.1.

Figure 16.1. SQL UNION operator

If the Customers view is updatable, the behavior of the view is the same as that of a table named Customers. The Customers view makes the location of the data transparent to applications. Applications reference the view instead of having to determine which member table contains the data. This transparency allows database administrators to repartition tables without recoding applications. In the event that one needs to repartition data, only the distributed partitioned view needs updating, a simple process.

Local partitioned views reference member tables on one server. Distributed partitioned views reference member tables on multiple servers. A server containing a member table is called a member server.

SQL Server version 7.0 and SQL Server 2000 support partitioned views; however, SQL Server 2000 introduces key features that allow the views to scale out. SQL Server 2000 partitioned views are updatable. This is crucial for distributing data so that the location of the data is transparent to the application. Updatable views support the full behavior of the original table; nonupdatable views are like read-only copies.

The SQL Server 2000 query optimizer supports new optimizations that minimize the amount of distributed data to be transferred. The distributed execution plans generated by SQL Server 2000 result in good performance for a larger set of queries than the plans generated by SQL Server version 7.0.

SQL Server 2000 partitioned views (distributed or local) are best suited for the types of SQL statements generated by Web sites and online transaction processing (OLTP) systems.

How Distributed Partitioned Views Work

SQL Server 2000 distributed partitioned views use distributed query functionality to extend the capabilities of partitioned views across multiple servers. Setting up distributed partitioned views is an easy, three-step process:

  1. Horizontally partition a table into member tables that have the same format as the original, are on separate servers, and have subsets of the original set of rows.

  2. On each member server, specify linked server definitions used to send distributed queries to each of the other member servers.

  3. On each member server, define a distributed partitioned view that combines rows from the member tables into a virtual table that behaves like the original table.

Whenever a SQL statement references a distributed partitioned view, the SQL Server 2000 query optimizer builds a dynamic query execution plan that pulls together the data from only those member tables containing data required by the statement.

Building Distributed Partitioned Tables and Views

The first step in implementing a partitioned view is to partition a table horizontally. The original table is replaced with several smaller member tables. Each member table has the same number of columns as the original table, and each column has the same attributes (data type, size) as the corresponding column in the original table.

The member tables are designed so that each table stores a horizontal slice of the original table based on a range of key values. The ranges are based on the data values in a partitioning column. The range of values in each member table is enforced by a CHECK constraint on the partitioning column, and ranges cannot overlap. For example, you cannot have one table with a range from 1 through 200 and another with a range from 150 through 300 because it would not be clear which table contains the values from 150 through 200. The CHECK constraint for the first CustomerID column in this example is

CustomerID   INTEGER PRIMARY KEY
             CHECK (CustomerID BETWEEN 1 AND 32999)

Next, a distributed partitioned view is defined on each member server; each view has the same name. This allows queries referencing the distributed partitioned view name to run on any of the member servers. The system operates as if a copy of the original table is on each member server, but each server has only a member table and a distributed partitioned view. The location of the data is transparent to the application.

To build the distributed partitioned views, follow these steps.

  1. Add linked server definitions on each member server containing the connection information required to execute distributed queries on the other member servers. This gives a distributed partitioned view access to data on the other servers.

  2. Create a distributed partitioned view on each member server. The views use distributed SELECT statements to access data from the linked member servers and merge the distributed rows with rows from the local member table. If the locations of the sample tables are Customers_33 on Server1, Customers_66 on Server2, and Customers_99 on Server3, on Server1 you would add a linked server definition named Server2 with the connection information for Server2, and a linked server definition named Server3 for access to Server3.

    CREATE VIEW Customers AS
   SELECT * FROM CompanyDatabase.TableOwner.Customers_33
UNION ALL
   SELECT * FROM Server2.CompanyDatabase.TableOwner.Customers_66
UNION ALL
   SELECT * FROM Server3.CompanyDatabase.TableOwner.Customers_99

  3. Perform the same steps on Server2 and Server3.

How the Query Processor Uses Distributed Partitioned Views

The Microsoft SQL Server 2000 query processor has been enhanced to optimize the performance of distributed partitioned views. The most important enhancement is minimizing the amount of data that must be transferred between member servers.

SQL Server 2000 builds intelligent, dynamic plans that make efficient use of distributed queries to access data from remote member tables. The query processor uses OLE DB to retrieve the CHECK constraint definitions from each member table to map the distribution of member table key values. It then compares the key ranges specified in an SQL statement to the member table key-range map and builds an execution plan that retrieves only those remote rows required to complete the SQL statement. The execution plan also delays any access to remote member tables until the information is required.

For example, assume this query was executed on Server1:

SELECT *
FROM Customers
WHERE CustomerID BETWEEN 3200000 AND 3400000

SQL Server 2000 extracts the rows with key values from 3200000 through 3299999 from the local table and uses a distributed query to retrieve the rows with key values from 3300000 through 3400000 from Server2.

The SQL Server 2000 query processor can also build dynamic logic into query execution plans for SQL statements where the key values are not known when the plan must be built. For example:

CREATE PROCEDURE GetCustomer @CustomerIDParameter INT AS
  SELECT *
  FROM CompanyDatabase.TableOwner.Customers
  WHERE CustomerID = @CustomerIDParameter

SQL Server 2000 cannot predict what parameter value will be supplied when the procedure is executed, so the query optimizer cannot predict which member table will have to be accessed. In this case, SQL Server builds an execution plan with conditional logic to control which member table is accessed. For example:

IF @CustomerIDParameter BETWEEN 1 and 3299999
  Get row in local table CompanyDatabase.TableOwner.Customer_33
ELSEIF @CustomerIDParameter BETWEEN 3300000 and 6599999
  Get row in linked table Server2.CompanyDatabase.TableOwner.Customer_66
ELSEIF @CustomerIDParameter BETWEEN 6600000 and 9999999
  Get row in linked table Server3.CompanyDatabase.TableOwner.Customer_99


Designing Systems That Maximize Distributed Partitioned View Performance

Distributed partitioned views were the key new feature that allowed Microsoft SQL Server 2000 to achieve the world-record performance levels reported in the launch of Microsoft Windows 2000. To achieve extremely high performance, a scaled-out system must balance the processing load across multiple servers in each tier. A federated database tier can do this best if the application sends each SQL statement to the member server having most of the data required by the statement. While the federated system offers data transparency to the application tier, including logic in the application tier for data dependent routing almost always dramatically improves performance.

Designing Partitions

Partitioning works well if the tables in the database are naturally divisible into similar partitions where most of the rows accessed by any SQL statement can be placed on the same member server. Tables are clustered in related units. For example, suppose the entry of an order references the Orders, Customers, and Parts tables, along with all tables that record the relationships between customers, orders, and parts. Partitions work best if all the rows in a logical cluster can be placed on the same member server.

Partitioning is most effective if tables in a database can be partitioned symmetrically where related data is placed on the same member server, so that most SQL statements routed to the correct member server will have minimal, if any, requirements for data on other member servers. A distributed partitioned view design goal can be stated as an 80/20 rule: Design partitions so that most SQL statements can be routed to a member server that has at least 80 percent of the data and distributed queries are required for 20 percent or less of the data. A good test of whether this rule can be applied is to see whether the partition allows all rows to be placed on the same member server as all their referencing foreign key rows. Database designs that support this goal are good candidates for partitioning.

The data is partitioned uniformly across the member servers. For example, suppose a company divides North America into regions. Each employee works in one region, and customers make most of their purchases in the state or province where they live. The region and employee tables are partitioned along regions. Customers are partitioned in regions by their state or province. Although some queries require data from multiple regions, the data required for most queries is on the server for one region. Applications route SQL statements to the member server containing the region inferred from the context of the user input.

While symmetric partitions are the ideal goal, most applications have complex data access patterns that prevent symmetrical partitioning. Asymmetric partitions result in some member servers assuming larger roles than others. For example, only some of the tables in a database may be partitioned, with the tables that have not been partitioned remaining on the original server. Asymmetric partitions can provide much of the performance of a symmetric partition, with two important benefits: (1) They dramatically improve the performance of a database that cannot be symmetrically partitioned by asymmetrically partitioning of some of its tables; (2) They successfully partition a large existing system by making a series of iterative, asymmetric improvements. The tables chosen for partitioning in each step are usually those that will give the highest performance gain at that time. In an asymmetric approach, the original server usually retains some tables that did not fit the partitioning scheme. The performance of these remaining tables is usually faster than the original system because the member tables move to member servers, reducing the load on the original server. The world-record benchmark results announced with the product launch of Windows 2000 were achieved in a system where one of the nine tables was not partitioned.

Many databases can be partitioned in more than one way. The specific partitions chosen for implementation must be those that best meet the requirements of the typical range of SQL statements executed by the business services tier.

Partitioning should also be designed in a way that produces routing rules, applications you can use to determine which member server can most effectively process each SQL statement. The business services tier must be able to match a piece of user data against the routing rules to find which member server processes the SQL statement.

Designing the Application Tier for Load Balancing

High-performance business services tiers are designed as identical COM+ components running on application servers. Windows 2000 Network Load Balancing can then evenly distribute user requests across the business tier. Because a user request can be processed on any application server, the business components must have some method for routing SQL statements to the appropriate member server. The business components should be able to match a data value received from the client against the data routing rules to determine which member server should process the request. As mentioned earlier, the federated database offers data transparency, but adding data-dependent routing logic to the application tier minimizes the work required by the database tier.

The most flexible solution involves storing the routing rules in a persistent store, such as the Active Directory service in Windows 2000, or in a database table. The business components retrieve the rules at runtime and use generic logic to compare a user data value against key distribution data stored in the data routing rules. Repartitioning the member tables requires no application changes; you simply adjust the routing rules.

Designing for High Availability

The data for a large Web site or internal OLTP system must be highly reliable. The data must be available 24 hours a day, 7 days a week, 52 weeks a year. In a shared-nothing clustered application tier, the loss of one server may degrade system performance, but it will not stop the entire system. The remaining servers in the cluster rebalance the load until a replacement server can be plugged into the cluster.

Although SQL Server 2000 does not support this type of load balanced clustering, it does support Microsoft Cluster Services failover clustering. In failover clustering, each database server is made of two to four physical servers that appear to applications as a single virtual server. If the primary server node fails, another node detects the loss of the primary and starts servicing all requests sent to the virtual server automatically. The cluster remains running until a replacement server is plugged into the cluster. Failover clustering helps provide high availability, but it does not perform any load balancing.


Backing Up and Restoring Federated Database Servers

In a federated database server tier built using distributed partitioned views, the member servers form one logical unit. The recovery of the member databases must be coordinated to ensure that they remain synchronized.

SQL Server 2000 does not require you to coordinate backups across member servers. Backups can be taken of each database independently, without regard for the state of the other member databases. Because the backups do not have to be synchronized, there is no processing overhead for synchronization and no blockage of running tasks.

SQL Server 2000 does have a method of simultaneously marking the system log files of all member databases with a named time in a coordinated fashion. In the event of a catastrophic failure, all the member systems may be restored to exactly the same time, preserving the transactional integrity of the complete federated system.

The most important aspect of recovering a set of member databases is planning and testing the recovery procedures before putting the databases in production.


Updatable Partitioned Views

If a local or distributed partitioned view is not updatable, it can serve only as a read-only copy of the original table. An updatable partitioned view can exhibit all the capabilities of the original table.

A view is considered an updatable partitioned view if the view is a set of SELECT statements whose individual result sets are combined into one using the UNION ALL statement. Each individual SELECT statement references one SQL Server base table. The table can be either a local table or a linked table referenced using a four-part name, the OPENROWSET function, or the OPENDATASOURCE function (you cannot use an OPENDATASOURCE or OPENROWSET function that specifies a pass-through query).

Table Rules

Member tables are defined in the FROM clause in each SELECT statement in the view definition. Each member table must adhere to these rules:

  • Member tables cannot be referenced more than once in the view.

  • Member tables cannot have indexes created on any computed columns.

  • Member tables must have all PRIMARY KEY constraints on an identical number of columns.

  • Member tables must have the same ANSI padding setting.

Column Rules

Columns are defined in the select list of each SELECT statement in the view definition. The columns must follow these rules.

  • All columns in each member table must be included in the select list.

  • The same column cannot be used multiple times in the select list.

  • Columns cannot be referenced more than once in the select list.

  • The columns must be in the same ordinal position in the select list.

  • The columns in the select list of each SELECT statement must be of the same type (including data type, precision, scale, and collation). For example, this view definition fails because the first column in both SELECT statements does not have the same data type:

    CREATE VIEW NonUpdatable
AS
SELECT IntPrimaryKey, IntPartNmbr
FROM FirstTable
  UNION ALL
SELECT NumericPrimaryKey, IntPartNmbr
FROM SecondTable

Partitioning Column Rules

A partitioning column exists on each member table and, through CHECK constraints, identifies the data available in that specific table. Partitioning columns must adhere to these rules:

  • Each base table has a partitioning column whose key values are enforced by CHECK constraints.

  • The key ranges of the CHECK constraints in each table do not overlap with the ranges of any other table. Any given value of the partitioning column must map to only one table. The CHECK constraints can only use these operators: BETWEEN, AND, OR, <, <=, >, >=, =.

  • The partitioning column is in the same ordinal location in the select list of each SELECT statement in the view. For example, the partitioning column is always the first column in each select list or the second column in each select list and so on.

  • Partitioning columns cannot allow nulls.

  • Partitioning columns must be a part of the primary key of the table.

  • Partitioning columns cannot be computed columns.

  • There must be only one constraint on the partitioning column. If there is more than one constraint, SQL Server ignores all the constraints and will not consider them when determining whether or not the view is a partitioned view.

A partitioned column that meets all these rules will support all the optimizations supported by the SQL Server 2000 query optimizer.

Data Modification Rules

In addition to the rules defined for updatable partitioned views, data modification statements referencing the view must adhere to the rules defined for INSERT, UPDATE and DELETE statements. (Note: You can modify data through a partitioned view only if you install the Enterprise or Developer Edition of Microsoft SQL Server 2000.)

INSERT Statements

INSERT statements add data to the member tables through the partitioned view. The INSERT statements must adhere to these rules:

  • All columns must be included in the INSERT statement even if the column can be NULL in the base table or has a DEFAULT constraint defined in the base table.

  • The DEFAULT keyword cannot be specified in the VALUES clause of the INSERT statement.

  • INSERT statements must supply a value that satisfies the logic of the CHECK constraint defined on the partitioning column for one of the member tables.

  • INSERT statements are not allowed if a member table contains a column with an identity property.

  • INSERT statements are not allowed if a member table contains a timestamp column.

  • INSERT statements are not allowed if there is a self-join with the same view or any of the member table.

UPDATE Statements

UPDATE statements modify data in one or more of the member tables through the partitioned view. The UPDATE statements must adhere to these rules:

  • UPDATE statements cannot specify the DEFAULT keyword as a value in the SET clause even if the column has a DEFAULT value defined in the corresponding member table.

  • The value of a column with an identity property cannot be changed, but the other columns can be updated.

  • The value of a PRIMARY KEY cannot be changed if the column contains text, image, or ntext data.

Updates are not allowed if a base table contains a timestamp column.

Updates are not allowed if there is a self-join with the same view or any of the member tables.

The DEFAULT keyword cannot be specified in the SET clause of the UPDATE statement.

DELETE Statements

DELETE statements remove data in one or more of the member tables through the partitioned view. DELETE statements are not allowed if there is a self-join with the same view or with any of the member tables.

Distributed Partition View Rules

In addition to the rules defined for partitioned views, distributed (remote) partition views have these additional conditions:

  • A distributed transaction is started to ensure atomicity across all nodes affected by the update.

  • The XACT_ABORT SET option must be set to ON.

  • Any linked server cannot be a loopback linked server, that is, a linked server that points to the same instance of SQL Server.

  • A view that references partitioned tables without following all these rules may still be updatable if there is an INSTEAD OF trigger on the view. The query optimizer, however, may not always be able to build execution plans for a view with an INSTEAD OF trigger that are as efficient as the plans for a partitioned view that follows all of the rules.


Summary

We covered the topic of distributed partitioned views in this chapter and covered the limitations and the advantages of creating DPVs, which can be helpful in the case of data warehousing applications, but with the current functionality it leaves a lot to be desired for making a successful implementation for the OLTP applications.


Chapter 17. T-SQL Debugger

SQL Query Analyzer comes equipped with a Transact-SQL debugger that allows you to control and monitor the execution of stored procedures. The debugger supports traditional functionality such as setting breakpoints, defining watch expressions, and single-stepping through procedures.

Table 17.1. T-SQL debugging options
Option | Description
Go | Runs the stored procedure in debugging mode.
Toggle Breakpoint | Sets or removes a breakpoint at the current line. Breakpoints cannot be set on lines containing nonexecutable code such as comments, declaration statements, or blank lines.
Remove All Breakpoints | Clears all breakpoints in your code.
Step Into | Executes one code statement at a time. Step Into executes the statement at the current execution point. If the statement is a call to a procedure, the next statement displayed is the first statement in the procedure.
Step Over | Executes one code statement at a time. If the current statement contains a call to a procedure, Step Over executes the procedure as a unit and then steps to the next statement in the current procedure. Therefore, the next statement displayed is the next statement in the current procedure regardless of whether the current statement is a call to another procedure.
Step Out | Executes the remaining lines of a function in which the current execution point lies. The next statement displayed is the statement following the procedure call. All the code is executed between the current and the final execution points.
Run to Cursor | Specifies a statement further down in the code where you want execution to stop. Use this option to avoid stepping through large loops.
Restart | Restarts execution from the beginning of the stored procedure.
Stop Debugging | Halts debugging.
Watch | Displays the current watch expressions.
Callstack | Lists the procedures calls that have started but are not completed.
Auto Rollback | Automatically rolls back all work performed during execution of the procedure.

It also supports debugging against Microsoft SQL Server 2000, SQL Server version 7.0, and SQL Server version 6.5 Service Pack 2.

The Transact-SQL Debugger window displays the text of the stored procedure to be debugged and provides the debugging options shown in Table 17.1.

The code pane displays the SQL statement you are debugging. Separate output panes are provided for local and global variables, for the result set of the query, and for the call stack. Note that "global variable" is a legacy term for the Transact-SQL built-in functions whose names start with @@.

Developers will appreciate this feature because they can now debug their stored procedures directly in the Query Analyzer and can troubleshoot the issues in T-SQL. You need to make sure that the debugger interface is installed at the time of installing SQL Server. If not, you can rerun setup and this option is available in the "Add components to your existing installation"; make sure that you select Development Tools\Debugger Interface.

You debug the procedure by giving into the object browser in Query Analyzer, drilling down to the stored procedure, right-clicking and clicking debug (Figure 17.1). Then you can specify the values for the parameters and set breakpoints to debug your code.

Figure 17.1. Debug option

Figure 17.2 is a snapshot of a scenario where a breakpoint is set and code is debugged to find the error. You can step through the code to analyze the issue.

Figure 17.2. Setting the breakpoint and debugging the code
    I l@ve RuBoard


Summary

In this chapter we talked about the Transact-SQL debugger functionality and the different options that are available in the debugger to troubleshoot issues. It is a very handy feature for finding out mistakes in your T-SQL code.


Chapter 18. Cascading Declarative Referential Integrity

This feature is new to SQL Server 7.0, although Oracle has had it for quite some time. Introduction of this feature helps in designing the database definitions more efficiently. You can control the actions SQL Server 2000 takes when you attempt to update or delete a key to which existing foreign keys point. This is controlled by the new ON DELETE and ON UPDATE clauses in the REFERENCES clause of the CREATE TABLE and ALTER TABLE statements.


Cascading Referential Integrity Constraints

Cascading referential integrity constraints allow you to define the actions Microsoft SQL Server 2000 takes when a user attempts to delete or update a key to which existing foreign keys point.

The REFERENCES clauses of the CREATE TABLE and ALTER TABLE statements support ON DELETE and ON UPDATE clauses:

[ ON DELETE { CASCADE | NO ACTION }  ]
[ ON UPDATE { CASCADE | NO ACTION }  ]

  • NO ACTION is the default if ON DELETE or ON UPDATE is not specified. NO ACTION specifies the same behavior that occurs in earlier versions of SQL Server.

  • ON DELETE NO ACTION specifies that if an attempt is made to delete a row with a key referenced by foreign keys in existing rows in other tables, an error is raised and the DELETE is rolled back.

  • ON UPDATE NO ACTION specifies that if an attempt is made to update a key value in a row whose key is referenced by foreign keys in existing rows in other tables, an error is raised and the UPDATE is rolled back.CASCADE allows deletions or updates of key values to cascade through the tables defined to have foreign key relationships that can be traced back to the table on which the modification is performed. CASCADE cannot be specified for any foreign keys or primary keys that have a timestamp column.

  • ON DELETE CASCADE specifies that if an attempt is made to delete a row with a key referenced by foreign keys in existing rows in other tables, all rows containing those foreign keys are also deleted. If cascading referential actions have also been defined on the target tables, the specified cascading actions are also taken for the rows deleted from those tables.

  • ON UPDATE CASCADE specifies that if an attempt is made to update a key value in a row, where the key value is referenced by foreign keys in existing rows in other tables, all the foreign key values are also updated to the new value specified for the key. If cascading referential actions have also been defined on the target tables, the specified cascading actions are also taken for the key values updated in those tables.

Examples of cascading referential actions can be based on the FK_Products_Suppliers constraint on the Products table in Northwind. This constraint establishes a foreign key relationship from the SupplierID column in the Products table to the SupplierID primary key column in the Suppliers table. If ON DELETE CASCADE is specified for the constraint, deleting the row in Suppliers where SupplierID equals 1 also deletes the three rows in Products where SupplierID equals 1. If ON UPDATE CASCADE is specified for the constraint, updating the SupplierID value in the Suppliers table from 1 through 55 also updates the SupplierID values in the three rows in Products whose SupplierID values currently equal 1.

Cascading actions cannot be specified for a table that has an INSTEAD OF trigger. After a cascading action has been defined for a table, an INSTEAD OF trigger cannot be added to it.


Multiple Cascading Actions

Individual DELETE or UPDATE statements can start a series of cascading referential actions. For example, a database contains three tables, TableA, TableB, and TableC. A foreign key in TableB is defined with ON DELETE CASCADE against the primary key in TableA. A foreign key in TableC is defined with ON DELETE CASCADE against the primary key in TableB. If a DELETE statement deletes rows in TableA, the operation also deletes rows in TableB that have foreign keys matching the deleted primary keys in TableA; it then deletes any rows in TableC that have foreign keys that match the deleted primary keys in TableB.

The series of cascading referential actions triggered by a single DELETE or UPDATE must form a tree containing no circular references. No table can appear more than once in the list of all cascading referential actions that result from the DELETE or UPDATE. The tree of cascading referential actions must not have more than one path to any given table. Any branch of the tree is terminated when it encounters a table for which NO ACTION has been specified or is the default.


Triggers and Cascading Referential Actions

Cascading referential actions fire AFTER triggers in this sequence:

  1. All the cascading referential actions directly caused by the original DELETE or UPDATE are performed first.

  2. When the original cascading referential actions have completed, the AFTER triggers on the original table are fired, regardless of whether any rows were updated.

  3. AFTER triggers on tables in the chain of cascaded referential actions are then fired, but only if one or more rows in the table have been updated or deleted.

If any errors are generated by any of the original set of cascading referential actions, an error is raised, no AFTER triggers are fired, and the DELETE or UPDATE is rolled back.

An AFTER trigger can execute a DELETE or UPDATE statement that starts another chain of cascading referential actions. Each secondary chain of referential actions is treated independently. These secondary chains of referential actions behave like the primary chain. All the secondary referential actions are completed before any secondary triggers are fired. Within each independent unit, there is no defined order in which the cascading referential actions are executed and the affected triggers are fired.

A table that has an INSTEAD OF trigger cannot also have a REFERENCES clause that specifies a cascading action. An AFTER trigger on a table targeted by a cascading action, however, can execute an INSERT, UPDATE, or DELETE statement on another table or view that fires an INSTEAD OF trigger defined on that object.


Cascading Referential Constraints Catalog Information

Catalog information about cascading referential constraints is shown in Table 18.1. The Transact-SQL OBJECTPROPERTY function supports these new values for the property parameter.

The REFERENTIAL_CONSTRAINTS information schema view returns CASCADE in the UDPATE_RULE or DELETE_RULE column when either ON UPDATE CASCADE or ON DELETE CASCADE is specified. NO ACTION is returned when either ON UPDATE NO ACTION or ON DELETE NO ACTION is specified, or if ON UPDATE or ON DELETE is not specified at all.

The UPDATE_RULE and DELETE_RULE columns returned by sp_fkeys and sp_foreignkeys are set to 1 when CASCADE is specified and return 0 when NO ACTION is specified or is the default.

Table 18.1. Catalog information about cascading referential constraints
Value | Object | Description
CnstIsDeleteCascade | Constraint | FOREIGN KEY constraint defined with ON DELETE CASCADE
CnstIsUpdateCascade | Constraint | FOREIGN KEY constraint defined with ON UPDATE CASCADE

Table 18.2. New columns in a result set when a foreign key is specified as the object of sp_help
Column name | Data type | Description
delete_action | nvarchar(9) | Indicates whether the delete action is CASCADE, NO ACTION, or N/A (not applicable).
update_action | nvarchar(9) | Indicates whether the update action is CASCADE, NO ACTION, or N/A (not applicable).

When a foreign key is specified as the object of sp_help, the output result set contains the new columns shown in Table 18.2.


Summary

In this chapter we talked about the new feature of cascading referential integrity, helpful while laying out your database structure.


Chapter 19. Collation Enhancements

In SQL Server 2000, you no longer need to specify the sort order and the code page. All this has now been replaced with collations that can be specified at the database level as well as the column level. So no longer do we need to uninstall SQL Server and reinstall it to select a different code page and sort order. Introducing collations in SQL Server 2000 also helps internationalize the custom products. A collation specifies the bit patterns that represent each character and the rules by which characters are sorted and compared.

SQL Server 2000 supports objects that have different collations being stored in a single database. Separate SQL Server 2000 collations can be specified down to the level of columns. Each column in a table can be assigned different collations. Earlier versions of SQL Server support only one collation for each instance of SQL Server. All databases and database objects created in an instance of SQL Server 7.0 or earlier have the same collation.


How Character Data Is Stored

In a computer, characters are represented by different patterns of ON and OFF bits. There are 8 bits in a byte, and the 8 bits can be turned ON and OFF in 256 different patterns. A program that uses 1 byte to store each character can therefore represent up to 256 different characters by assigning a character to each of the bit patterns. There are 16 bits in 2 bytes, and 16 bits can be turned ON and OFF in 65,536 unique patterns. A program that uses 2 bytes to represent each character can represent up to 65,536 characters.

Single-byte code pages are definitions of the characters mapped to each of the 256 bit patterns possible in a byte. Code pages define bit patterns for uppercase and lowercase characters, digits, symbols, and special characters such as !, @, #, or %. Each European language, such as German or Spanish, has its own single-byte code page. Although the bit patterns used to represent the Latin alphabet characters A through Z are the same for all the code pages, the bit patterns used to represent accented characters such as é and á vary from one code page to the next. If data is exchanged between computers running different code pages, all character data must be converted from the code page of the sending computer to the code page of the receiving computer. If the source data has extended characters that are not defined in the code page of the receiving computer, data is lost. When a database serves clients from many different countries, it is difficult to pick a code page for the database that contains all the extended characters required by all the client computers. Also, a lot of processing time is spent doing the constant conversions from one code page to another.

Single-byte character sets are also inadequate to store all the characters used by many languages. For example, some Asian languages have thousands of characters and so must use two bytes per character. Double-byte character sets have been defined for these languages. Still, each of these languages has its own code page, and there are difficulties in transferring data from a computer running one double-byte code page to a computer running another.

SQL Server 2000 supports the following code pages.

1258 | Vietnamese
1257 | Baltic
1256 | Arabic
1255 | Hebrew
1254 | Turkish
1253 | Greek
1252 | Latin1 (ANSI)
1251 | Cyrillic
1250 | Central European
950 | Chinese (Traditional)
949 | Korean
936 | Chinese (Simplified)
932 | Japanese
874 | Thai
850 | Multilingual (MS-DOS Latin1)
437 | MS-DOS U.S. English

To address the character conversion and interpretation problems that occur when trying to support multiple code pages in a network, the International Standards Organization and a group called the Unicode Consortium defined the Unicode standard. Unicode uses two bytes to store each character. Because 65,536 characters are enough to cover all the commonly used characters in all the languages of the world, all major languages are covered by the Unicode standard. If all the computers and programs in a network use Unicode, character conversions are not needed; each user will see exactly the same characters as all other users, and no loss of characters will occur.

On computers running Microsoft Windows operating systems, the code page used by the operating system and Windows applications is defined by the Windows locale. The locale is selected when the operating system is installed. Windows applications interpret character data using the code page defined by the Windows locale. Windows applications also support wide-character, or Unicode, data.

SQL Server 2000 supports two categories of character data types; the Unicode data types nchar, nvarchar, and ntext and the non-Unicode character data types char, varchar, and text. The Unicode data types use the Unicode character representation. Code pages do not apply to these data types. The non-Unicode data types use the character representation scheme defined in a single- or double-byte code page.

For more information about how character data is stored and the operation of code pages, Unicode, and sort orders, see "Developing International Software for Windows 95 and Windows NT 4.0" on the MSDN page at http://msdn.microsoft.com.


International Data and Unicode

Storing data in multiple languages within one database is difficult to manage when using only character data and code pages. It is difficult to find one code page for the database that can store all the required language-specific characters. It is also difficult to ensure the proper translation of special characters when being read or updated by different clients running various code pages. Say a database of customers in North America has to handle three major languages: Spanish names and addresses for Mexico, French names and addresses for Quebec, and English names and addresses for the rest of Canada and the United States. If only character columns and code pages are used, care has to be taken to ensure that the database is installed with a code page that will handle the characters of all three languages. More care must be taken to ensure the proper translation of characters from one of the languages when read by clients running a code page for another language.

With the growth of the Internet, it is becoming more important than ever before to support many client computers running different locales. It is difficult to pick a code page for character data types that will support all the characters required by a worldwide audience.

The easiest way to manage character data in international databases is to always use the Unicode nchar, nvarchar, and ntext data types, their non-Unicode equivalents (char, varchar, and text). If all the applications that work with international databases also use Unicode variables instead of non-Unicode variables, character translations do not have to be performed anywhere in the system. All clients see exactly the same characters in data as all other clients.

For systems that could use single-byte code pages, the fact that Unicode data needs twice as much storage space as non-Unicode character data is at least partially offset by eliminating the need to convert extended characters between code pages. Systems using double-byte code pages do not have this issue.

SQL Server 2000 stores all textual system catalog data in columns having Unicode data types. The names of database objects such as tables, views, and stored procedures are stored in Unicode columns. This allows applications to be developed using only Unicode, which avoids all issues with code page conversions.


Sort Order

A sort order specifies the rules used by SQL Server to interpret, collate, compare, and present character data. For example, a sort order defines whether the letter a is less than, equal to, or greater than the letter b. A sort order defines whether the collation is case-sensitive, for example whether lowercase m is equal or not equal to uppercase M. It also defines if the collation is accent-sensitive, for example whether á is equal or not equal to ä.

SQL Server 2000 uses two sort orders with each collation, one for Unicode data and another for the character code page. Many SQL Server collations use the same code page but have a different sort order for the code page. This allows sites to choose: whether characters will be sorted simply based on the numeric value represented by their bit patterns or the values in the code page. Binary sorting is fastest because SQL Server does not have to make any adjustments and can use fast, simple sorting algorithms. Binary sort orders are always case-sensitive. Because the bit patterns in a code page may not be arranged in the same sequence as defined by the dictionary rules for a specific language, binary sorting sometimes does not sort characters in a sequence users who speak that language might expect.


What Is Unicode and How Can It Be Used?

Unicode support is the foundation of the multilingual support of SQL Server 2000. Unicode is a standard that is designed to support all the world's scripts. Unicode supplies a unique code point for every character, regardless of platform, program, or language. A program that supports Unicode can handle data in any language. Unicode 3.0 can handle up to 1,114,112 characters.

Unicode is an industry standard managed by the Unicode Consortium, an organization that recognizes the importance of having a single character set for all languages. Microsoft is a member of the Unicode Consortium. Most companies joined for many of the same reasons that Microsoft did: In creating global software solutions, the importance of being able to represent multilingual data is obvious. Many other companies and individuals join to understand the issues and techniques for handling multilingual data.

The Unicode Standard, now at version 3.01, is identical to ISO-10646, an international standard that matched all the code points in Unicode just after Unicode 1.1 was released. The effective combination of both an industry and an international standard helps to keep individual interests from subverting the goal of both standards: one character set for everyone!

Unicode maps code points to characters but do not actually specify how the data will be represented in memory, in a database, or on a Web page. This is where the actual encoding of Unicode data comes into play. There are many different encodings for Unicode. This section describes these common encodings:

  • UCS-2

  • UTF-16

  • UTF-8

The information about encoding is presented here to help you better understand Unicode and some of the many ways in which it can be stored. Most of the time you can simply choose a Unicode data type and not worry about these details; however, it is important to understand the encoding any time you are dealing with an application that may encode Unicode differently, must send data to other platforms (non-Microsoft Windows) or Web servers, or must manage the importation of data from or exportation of data to other encodings.

UCS-2

UCS-2 is the main Unicode encoding used by Microsoft Windows NT 4.0, Microsoft SQL Server version 7.0, and Microsoft SQL Server 2000. UCS-2 allows for the encoding of 65,536 different code points. All information that is stored in Unicode in SQL Server 2000 is stored in this encoding, which uses two bytes for every character, regardless of the character being used. Therefore, the Latin letter A is treated the same way as the Cyrillic letter sha or the Hebrew letter lamed. Each one has a unique code point (for these letters, the code points are U+0041, U+0248, U+05DC, U+0BB1, and U+3048, respectively, where each four-digit hexadecimal number represents the 2 bytes that UCS-2 uses).

The ordering of bytes can be crucially important at the operating-system level. Because SQL Server runs on the Windows platform, it uses a Little Endian encoding system. Therefore, a hexadecimal word such as 0x1234 is stored in memory as 0x34 0x12.

UTF-16

UTF-16 is the primary Unicode encoding used by Microsoft Windows 2000. Even before Unicode 2.0 was released, it became clear that the goal of Unicode (to support a single code point for every character in every language) could not be achieved using only 65,536 code points. Some languages, such as Chinese, require that many code points to encode just the rarely used characters. Thus, support was added for a surrogate range to handle an additional 1,048,576 code points. UTF-16 is the encoding that fully supports this extension of the original standard. In UTF-16, the same standard of 2 bytes per code point is followed; however, with UTF-16 certain code points use another code point right after them to define the character. Like UCS-2, UTF-16 is stored in a Little Endian manner, as is everything on Windows by default.

Important: Although UCS-2 is not aware of surrogates, it will not corrupt the actual data in a database that contains them; it will treat them as two separate (undefined) characters.

Although SQL Server 7.0 and SQL Server 2000 can store surrogate pairs losslessly, they will treat the surrogate pairs as two undefined Unicode characters rather than as a single character. Such applications are usually referred to as "surrogate-neutral" or "surrogate-safe" (where safe refers to the ability to store the data even if there is no intrinsic ability to interact with it). "Surrogate-aware" applications are rare at this time, because currently there are no surrogate characters officially defined. Microsoft Word 2000, Microsoft Windows 2000, and Microsoft Internet Explorer version 5.0 and later are a few surrogate-aware applications.

UTF-8

Many ASCII and other byte-oriented systems that require 8-bit encodings (such as mail servers) must span a vast array of computers that use different encodings, different byte orders, and different languages. UTF-8 is an encoding scheme that is designed to treat Unicode data in a way that is independent of the byte ordering on the computer. Although SQL Server 2000 does not store data in UTF-8 format, it supports UTF-8 in at least one crucial scenario: its support of the Extensible Markup Language (XML). Many other database systems (such as Oracle and Sybase SQL Server) support Unicode using UTF-8 storage. Depending on a server's implementation, this can be technically easier for a database engine to implement (all the existing text management code on the server that is designed to deal with data 1 byte at a time does not require major changes). In the Windows environment, UTF-8 storage has these disadvantages:

  1. The Component Object Model (COM) supports only UTF-16/UCS-2 in its APIs and interfaces, which would require constant conversion if data were stored in UTF-8 format (this issue applies only when COM is used; SQL Server database engine does not typically call COM interfaces).

  2. The Windows NT and Windows 2000 kernels are both Unicode and use UCS-2 and UTF-16, respectively. Once again, a UTF-8 storage format would require many extra conversions. (As with the previous note on COM, this would not result in a conversion hit in the SQL Server database engine, but it would potentially affect many client-side operations.)

  3. UTF-8 can be slower for many string operations. Sorting, comparing, and virtually any string operation can be slowed because characters do not have a fixed width.

  4. UTF-8 often needs more than 2 bytes, and the increased size can make for a larger footprint on disk and in memory.

Because XML is, among other things, a very important standard for communication over the Internet (which has a strong byte-oriented bias), its default to UTF-8 is something that can make a lot of sense.


Data Types in SQL Server 2000

The central task of a database is obviously the storage of data. This section covers some issued involved in using SQL Server 2000 data types for storing international data.

Non-Unicode Text Types: char, varchar, text

When dealing with text data that is stored in the char, varchar, or text data types, the most important limitation to consider is that only information from a single code page can be stored. The exact code page depends on the collation of the column (if there is no column-level collation, then the collation of the database is used). To determine the code page that is used for a given column, the COLLATIONPROPERTY function can be used, as shown in the following examples:

SELECT COLLATIONPROPERTY('Chinese_PRC_Stroke_CI_AI_KS_WS', 'CodePage')
936
SELECT COLLATIONPROPERTY('Latin1_General_CI_AI', 'CodePage')
1252
SELECT COLLATIONPROPERTY('Hindi_CI_AI_WS', 'CodePage')
0

Hindi is given as an example to point out that many locales (such as Georgian and Hindi) do not have code pages, as they are "Unicode only" collations. Those collations are not appropriate for these data types. When Unicode data must be inserted into these columns, the columns are internally converted from Unicode using the WideCharToMultiByte API and the code page associated with the collation. When a character cannot be represented on the given code page, it is replaced by a question mark (?); this makes random question marks a good indication of data that has been corrupted due to conversion. It also is a good indication that a Unicode data type was needed. A string literal of a non-Unicode type is converted first using the database's default code page (derived from its collation).

Another problem might be encountered in trying to store data when not all of the characters in the data are contained in the code page. One of the best examples of this is Arabic script: It supports a wide array of languages, including Baluchi, Berber, Farsi, Kashmiri, Kazakh, Kirghiz, Pashto, Sindhi, Uighur, Urdu, and more. All these languages have additional characters beyond those in the Arabic language, which is the basis for Windows code page 1256. These extra characters are thus converted into question marks if they are stored in a non-Unicode column with the Arabic collation. This problem occurs because, in many cases, Windows considers a particular code page to be a "best fit" code page. That means that it may not be able to handle all text, but it is the best one available.

Unicode Text Types: nchar, nvarchar, ntext

The SQL-92 specification defines these "n" (stands for national) data types but does not specifically require them to be used for Unicode; the actual definition of these data types is left to the database platform or developer. In SQL Server 7.0 and SQL Server 2000, these data types are defined as being equivalent to UCS-2/UTF-16 Unicode. It is important to keep in mind that this definition is specific to Microsoft SQL Server. With other database servers (such as Sybase SQL Server), the "n" data types do not specifically mean Unicode.

For the storage of complex scripts, such as Hindi and Tamil, it is important to note that the data is expected to be in the proper order. Many languages such as Tamil actually specify that certain letters must be reordered when the text is rendered, thus making the logical order of text as it is stored in memory different from the visual order that is seen in a user interface. Data should always be stored in the proper logical order for any complex script language, which includes all of the Indic languages, Arabic, Farsi, Hebrew, and many others. The actual rendering of such data is a separate issue.

Although the "n" columns can indeed support data of any language or combination of languages, the actual sorting of this data can be in only a single collation. None of the code page limitations that were mentioned previously in this article apply to Unicode columns.

Date/Time Types: datetime, smalldatetime

The actual data types have no actual international meaning; they represent a date/time value with the following definitions:

datetime: Date and time data in the Gregorian calendar from January 1, 1753, through December 31, 9999, to an accuracy of one three-hundredth of a second (equivalent to 3.33 milliseconds or 0.00333 seconds).

smalldatetime: Date and time data in the Gregorian calendar from January 1, 1900, through June 6, 2079, with accuracy to the minute. smalldatetime values with 29.998 seconds or lower are rounded down to the nearest minute; values with 29.999 seconds or higher are rounded up to the nearest minute.

Microsoft SQL Server rejects data that falls outside these ranges. The actual data is stored internally as two integers (4-byte integers for datetime, 2-byte integers for smalldatetime) that represent the date and time in question. Because the actual value has no actual connotations in regard to locale-specific formatting, it is up to the developer to define such conversions as needed.

SQL Server 2000 supports many different locale-specific conversions that can be performed at the server instead of relying on custom solutions from developers. These date styles can be accessed through the CONVERT function, which takes a data type, an expression, and an optional style, as shown in Table 19.1.

The following example shows how the CONVERT function is used:

SELECT CONVERT(char, GETDATE(), 100) AS [100]
Aug 16 2000 11:50AM

Table 19.1. Convert function values
W/century | W/o century | Standard | Input (converting to datetime) Output (converting to text)
0 or 100 — Default mon dd yyyy hh:miAM (or PM)
101 | 1 | USA | mm/dd/yy
102 | 2 | ANSI | yy.mm.dd
103 | 3 | British/French | dd/mm/yy
104 | 4 | German | dd.mm.yy
105 | 5 | Italian | dd-mm-yy
106 6 — dd mon yy
107 7 — Mon dd, yy
108 8 — hh:mm:ss
9 or 109 — Default + mon dd yyyy hh:mi:ss:mmmAM (or PM)
milliseconds
110 | 10 | USA | mm-dd-yy
111 | 11 | JAPAN | yy/mm/dd
112 | 12 | ISO | yymmdd
13 or 113 — Europe default + dd mon yyyy hh:mm:ss:mmm(24h)
milliseconds
114 14 — hh:mi:ss:mmm(24h)
20 or 120 — ODBC canonical yyyy-mm-dd hh:mi:ss(24h)
21 or 121 — ODBC canonical + yyyy-mm-dd hh:mi:ss.mmm(24h)
milliseconds
126 — ISO8601 (no spaces) yyyy-mm-dd Thh:mm:ss:mmm
130 — Kuwaiti (Hijri) dd mon yyyy hh:mi:ss:mmmAM
131 — Kuwaiti (Hijri) dd/mm/yy hh:mi:ss:mmmAM

You can then convert the data from a string to a date value in much the same manner:

SELECT CONVERT(datetime, 'Aug 16 2000 11:50AM', 100) AS [100]

It is worth noting that when you convert such dates with Style 130 (Kuwaiti or Hijri), converting to the char data type may result in the data corruption if the collation is not one of the Arabic collations that use code page 1256 for Unicode conversions.

Note that on a U.S. client computer, attempting to use the char data type results in the Arabic characters being converted into question marks and the nchar data type rendering Arabic characters.


Collation in SQL Server 2000

In SQL Server 2000, the collation model has changed because (1) the requirement for two different collations was confusing, and (2) a more flexible model was needed to handle all the new places for which a collation could be specified. In SQL Server 2000, collations are also used to handle the code page of non-Unicode columns; therefore, more collations are required.

A single, consistent model was designed to handle both Unicode and non-Unicode sorts. This model supports the following languages:

Albanian | Arabic | Chinese_PRC
Chinese_PRC_Stroke | Chinese_Taiwan_Bopomofo | Chinese_Taiwan_Stroke
Cyrillic_General | Croatian | Czech
Danish_Norwegian | Estonian | Finnish_Swedish
French | Georgian_Modern_sort | German_PhoneBook
Greek | Hebrew | Hindi
Hungarian | Hungarian_Technical | Icelandic
Japanese | Japanese_Unicode | Korean_Wansung
Korean_Wansung_Unicode | Latin1_General | Latvian
Lithuanian | Lithuanian_Classic | FYRO Macedonian
Modern_Spanish | Polish | Romanian
Slovak | Slovenian | Thai
Traditional_Spanish | Turkish | Ukrainian
Vietnamese |  |

Each of these collations is combined with a series of suffixes that help define whether there is case, accent, width, or kana sensitivity. The exact suffixes possible are shown in Table 19.2. Each of the 40 languages in the previous list supports the 17 suffixes in the following table, for a total of 680 Windows collations.

These language names are arbitrary and were chosen to properly represent each unique supported code page for non-Unicode data and sort order for all data. In the many cases where a language can be completely represented on another code page or when a sort order that one language needs is covered by another, the language was "removed" from the list because it is adequately covered. Note that the default setting of kana sensitivity and width sensitivity is set to insensitive.

To ensure that code pages from earlier versions of SQL Server are properly supported, many backward-compatible, SQL-specific sort orders are also included in SQL Server 2000. These SQL-specific sort orders follow. Many of them support some of the various parts of the suffixes described previously, but not all the suffixes are supported.

SQL_1xCompat_CP850 | SQL_Estonian_CP1257 | SQL_Latin1_General_Pref_CP437
SQL_AltDiction_CP1253 | SQL_Hungarian_CP1250 | SQL_Latin1_General_Pref_CP850
SQL_AltDiction_CP850 | SQL_Icelandic_Pref_CP1 | SQL_Latvian_CP1257
SQL_AltDiction_Pref_CP850 | SQL_Latin1_General_CP1 | SQL_Lithuanian_CP1257
SQL_Croatian_CP1250 | SQL_Latin1_General_CP1250 | SQL_MixDiction_CP1253
SQL_Czech_CP1250 | SQL_Latin1_General_CP1251 | SQL_Polish_CP1250
SQL_Danish_Pref_CP1 | SQL_Latin1_General_CP1253 | SQL_Romanian_CP1250
SQL_EBCDIC037_CP1 | SQL_Latin1_General_CP1254 | SQL_Scandinavian_CP850
SQL_EBCDIC273_CP1 | SQL_Latin1_General_CP1255 | SQL_Scandinavian_Pref_CP850
SQL_EBCDIC277_CP1 | SQL_Latin1_General_CP1256 | SQL_Slovak_CP1250
SQL_EBCDIC278_CP1 | SQL_Latin1_General_CP1257 | SQL_Slovenian_CP1250
SQL_EBCDIC280_CP1 | SQL_Latin1_General_CP437 | SQL_SwedishPhone_Pref_CP1
SQL_EBCDIC284_CP1 | SQL_Latin1_General_CP850 | SQL_SwedishStd_Pref_CP1
SQL_EBCDIC285_CP1 | SQL_Latin1_General_Pref_CP1 | SQL_Ukrainian_CP1251
SQL_AltDiction_CP1253 | SQL_Hungarian_CP1250 |
SQL_Latin1_General_Pref_CP850 |  |

You can retrieve the actual information about collations by using the COLLATIONPROPERTY function. In addition to the CodePage value used earlier, you can also pass other information types, such as LCID, which returns the Windows Locale ID (returns Null for SQL collations). You can also specify Windows ComparisonStyle (returns Null for both binary and SQL collations). This information can be used to verify that there is indeed an equivalence between string normalization in Windows 2000 and SQL Server 2000 for all the Windows collations.

All the available collations can be returned by using the fn_helpcollations() function, for example:

Table 19.2. Suffixes for collation
Suffix for collation | Meaning
_BIN | Binary sort
_CI_AI | Case-insensitive, accent-insensitive, kanatype-insensitive, width-insensitive
_CI_AI_WS | Case-insensitive, accent-insensitive, kanatype-insensitive, width-sensitive
_CI_AI_KS | Case-insensitive, accent-insensitive, kanatype-sensitive, width-insensitive
_CI_AI_KS_WS | Case-insensitive, accent-insensitive, kanatype-sensitive, width-sensitive
_CI_AS | Case-insensitive, accent-sensitive, kanatype-insensitive, width-insensitive
_CI_AS_WS | Case-insensitive, accent-sensitive, kanatype-insensitive, width-sensitive
_CI_AS_KS | Case-insensitive, accent-sensitive, kanatype-sensitive, width-insensitive
_CI_AS_KS_WS | Case-insensitive, accent-sensitive, kanatype-sensitive, width-sensitive
_CS_AI | Case-sensitive, accent-insensitive, kanatype-insensitive, width-insensitive
_CS_AI_WS | Case-sensitive, accent-insensitive, kanatype-insensitive, width-sensitive
_CS_AI_KS | Case-sensitive, accent-insensitive, kanatype-sensitive, width-insensitive
_CS_AI_KS_WS | Case-sensitive, accent-insensitive, kanatype-sensitive, width-sensitive
_CS_AS | Case-sensitive, accent-sensitive, kanatype-insensitive, width-insensitive
_CS_AS_WS | Case-sensitive, accent-sensitive, kanatype-insensitive, width-sensitive
_CS_AS_KS | Case-sensitive, accent-sensitive, kanatype-sensitive, width-insensitive
_CS_AS_KS_WS | Case-sensitive, accent-sensitive, kanatype-sensitive, width-sensitive

SELECT * FROM ::fn_helpcollations()

This query returns 753 rows in SQL Server 2000. Additional collations cannot be added, unless they are added in service packs or future versions.

It is important to briefly explain how collations actually operate on Unicode data. As a general rule that has no exception, every defined collation in SQL Server on a Unicode column sorts all defined Unicode characters. There are many different collations because there are many differences in how the data can be sorted. A good example is the Georgian modern sort. Although the traditional sorting of Georgian text places all the characters in a specific order, it is common for modern usage to place certain rarely used characters at the end.

This does not stop any other Unicode data from being sorted according to the same sort provided in the Latin1_General collation. In fact, all collations sort Georgian in the traditional form, with the sole exception of the Georgian_Modern_Sort collations. The same rule applies to all other collations; only the exceptions change between collations.


Collations Specified at Multiple Levels

In SQL Server 2000, collations can be specified at the server level, at the database level, at the column level, or in an expression.

Collations at the Server Level

The server level is where the collation always used to be, and in many cases is the only collation you ever need to set. This collation acts as the default for all collations in all databases on the server when they are created, if you do not explicitly set the collation at the database level. Because a database is always given a collation, the server-level collation is never really consulted except when the database is created.

You can change this collation without rerunning Setup by using the Rebuild Master utility (RebuildM.exe), located in Program Files\Microsoft SQL Server\80\Tools\BINN directory. You can also query the server for the collation with the Transact-SQL SERVERPROPERTY function, for example:

SELECT CONVERT(char, SERVERPROPERTY('collation'))

Collations at the Database Level

Every database can have a unique collation, with the sort order being set at the database level. Figure 19.1 shows how the collation is set using SQL Server Enterprise Manager. You can set the collation order with Transact-SQL, as well. For example, to create a new database in Czech Republic sort order, case and accent sensitive, use a statement such as

USE master
GO
CREATE DATABASE Products
ON
( NAME = products_dat,
   FILENAME = 'c:\program files\microsoft sql
server\mssql\data\products.mdf' )
COLLATE Czech_CS_AS
GO
Figure 19.1. Setting collation using the Enterprise Manager

Interestingly, you can even change the collation of an existing database using the ALTER DATABASE statement (not available using SQL Server Enterprise Manager). For example, the following statement changes the collation of the Products database from Czech_CS_AS to Czech_CI_AI (case and accent sensitive to case and accent insensitive):

ALTER DATABASE Products
COLLATE Czech_CI_AI

Collations at the Column Level

In SQL Server 2000, you can change the collation of text in a particular column. This can be very useful, for example, in a circumstance when you might need to force case sensitivity for a password column. Different language columns would be useful in other scenarios. For example, a customer name might need to be in Unicode using Latin1_General for the broadest appropriate sorting, and a product line might always be in Greek, in which case a Greek collation might make sense.

Figure 19.2 displays a collation specification during table design, using SQL Server Enterprise Manager. When you click on the "…" button, the dialog box shown in Figure 19.3 appears. In this dialog box, you can choose a collation.

Figure 19.2. Specifying collation during table design using the Enterprise Manager

Figure 19.3. Collation dialog box

You can also set column-level collations using Transact-SQL. In the CREATE TABLE statement, simply add a COLLATE clause to the column definition. In the following example, the job description has a collation set for Arabic (case and accent insensitive, kanatype insensitive).

CREATE TABLE jobs
(
   job_id  smallint
      IDENTITY(1,1)
      PRIMARY KEY CLUSTERED,
   job_desc varchar(50)
      COLLATE Arabic_CI_AI_KS
      NOT NULL
      DEFAULT 'New Position - title not formalized yet',
)

You can use the ALTER TABLE statement to change the collation at the column level (except for an ntext or text column) with a new data type. However, you can change the collation on an ntext column using SQL Server Enterprise Manager. This is because SQL Server Enterprise Manager creates a temp table, moves data to the new temp table, drops the old table, creates a new table with the new collation, and then copies the data back to the new table.

Collations in Expressions

There may be many instances when you need to display data to people in different countries and want locale-appropriate sorting. With SQL Server 2000, you can specify collations in expressions. This powerful new feature allows you to sort in a particular manner so that the ORDER BY clause can be language-specific.

For example, the following query sorts the Customers table by last name and first name. (Using the Lithuanian sort order provides a good example of collation differences because the rules about how the letter Y sorts after the letter are so striking and noticeable.)

SELECT
  *
FROM
  tblCustomers
ORDER BY
  LastName COLLATE Lithuanian_AI_CI,
  FirstName COLLATE Lithuanian_AI_CI

You can also refer to the examples in SQL Server Books Online for SQL Server 2000 that relate to regular comparisons, such as

SELECT
  *
FROM
  Table1
WHERE
  Field1 = Field2 COLLATE Turkish_ci_ai

Considerations Before Changing the Collation of a Database

To change the collation of a database, the following must be true:

  1. No one else can be using the database.

  2. No schema-bound object can be dependent on the database collation. Schema-bound objects that qualify include any of the following:

    • User-defined functions and views created with SCHEMABINDING

    • Computed columns

    • CHECK constraints

    • Table-valued functions that return tables with character columns with collations inherited from the default database collation

The act of trying to change the collation of the database does not result in duplicates among any of the system names. This is very easy to imagine, for example, in the case of trying to change the collation from French_CI_AS to French_CS_AS (case insensitive to case sensitive). With the collation in earlier versions of SQL Server, it is possible to have two tables named Table1 and TABLE1; whereas in the collation in SQL Server 2000, this causes duplicates. Objects that can potentially cause such duplications include:

  • Object names (such as procedure, table, trigger, or view)

  • Schema names (such as group, role, or user)

  • Scalar-type names (such as system and user-defined types)

  • Full-text catalog names

  • Column or parameter names within an object

  • Index names within a table

None of these restrictions is unrealistic, and they keep users from doing things that can corrupt data or databases. In fact, some might think the rules are not strict enough! If there is data in a text, varchar, or char field and there is no explicit collation on the column, changing the collation of the database alters the way the data's encoding is interpreted, resulting in a form of corruption of any characters beyond the ASCII range (which is contained by all code pages). Rather than tempting fate in this way, changing the collation of any database that contains text data columns that are not one of the Unicode types should be avoided, unless those columns have their own explicit collations set.

The Transact-SQL DATABASEPROPERTYEX function can also be used to find out the collation of a database, for example:

SELECT CONVERT(char, DATABASEPROPERTYEX('pubs', 'collation'))


Collation Precedence

Assuming that Table1 has no explicit column-level collations, both columns are compared with the Turkish sort order. This is because of the collation precedence.

Collation precedence, also known as collation coercion rules, is the term given to the set of rules that determine: the collation of the final result of an expression that is evaluated to a character string and the collation used by collation-sensitive operators that use character string inputs but do not return a character string, such as LIKE and IN.

The collation precedence rules apply only to the character string data types, char, varchar, text, nchar, nvarchar, and ntext. Objects with other data types do not participate in collation evaluations.

In SQL Server 2000, you can specify a collation at the server, in the database, in the column, and in an expression. How do these collations interact?

The interaction works with some straightforward rules that may seem confusing at first. Table 19.3 shows an interaction in which A and B act as the two different parts of the comparison.

As you can see, the only instances in which SQL Server cannot handle an expression is when two different, conflicting collations are explicitly defined or when it tries to compare two items and can find no common ground for the comparison. These are not really blocking limitations; they are understandable rules. SQL Server simply needs some basis for comparison!

For example, consider the following Transact-SQL statement for creating a table:

CREATE TABLE TestTab (
   id int,
   GreekCol nvarchar(10) COLLATE greek_ci_as,
   LatinCol nvarchar(10) COLLATE latin1_general_cs_as
   )
INSERT TestTab VALUES (1, N'A', N'a')
GO

This statement creates a table with one column using a case-insensitive, accent-sensitive Greek collation and the other column using a case-sensitive, accent-sensitive General Latin1 collation.

Table 19.3. Collation interaction
Explicit B | Implicit B | Default | No Collation
Explicit A[1] | Run time error | Explicit A | Explicit A | Explicit A
Implicit A[2] | Explicit B | No collation | Implicit A | No collation
Default[3] | Explicit B | Implicit B | Default | No collation
No collation[4] | Explicit B | No collation | No collation | No collation

[1] Explicit A/Explicit B: A collation is explicitly defined for a given expression.

[2] Implicit A/Implicit B: A collation has been defined at the column level.

[3] Default: The database-level collation is being used.

[4] No Collation: There is a conflict between the two operators; the expression will be handled with no collation.

You could attempt to use a query to explicitly compare the two:

SELECT *
FROM TestTab
WHERE GreekCol = LatinCol

However, this returns an error:

Msg 446, Level 16, State 9, Server ROORKEE, Line 1
Cannot resolve collation conflict for equal to operation.

This occurs because the server cannot compare the two segments of text with different collations. If, however, you use the COLLATE keyword to explicitly create an expression that allows them to be compatible, the query will work like this:

SELECT *
FROM TestTab
WHERE GreekCol = LatinCol COLLATE greek_ci_as

You will also want to note that although LatinCol usually has a case-sensitive collation, the case-insensitive collation of the expression overrides this and allows the uppercase and lowercase A to be treated as equal.


COLLATE Keyword

The syntax for using the COLLATE keyword is

COLLATE [<Windows_Collation_name>|<SQL_Collation_Name]

The choice of collation name can be a difficult one to make. The keyword can be specified at the database level, the column level, or in expressions. As a rule, any time the field is not a Unicode one (ntext, nvarchar, or nchar) the earlier process in which the collation is converted into a code page is used.

There are two types of collations, Windows collations and SQL collations. Windows collations are defined by Windows. You have full options to specify case, accent, kana, and width sensitivity, as well as being able to define binary sorts.

SQL collations, are defined by SQL Server for legacy reasons. You do not have full options to configure these sorts. Generally, you should try to use the Windows collations whenever you can.


Limitations of the COLLATE Keyword

The COLLATE keyword and its related features are pretty incredible, and I believe they are without comparison among contemporary enterprise database products. There are some limitations all of which, however, have workarounds. The limitations are described here to help you understand what you can do directly and what requires a little extra work.

The fn_helpcollations function returns a full list of collations. However, SQL Server clearly can list one locale (such as Albanian) and provide the rest of the flags as options, returning the full string in the end. If you want to provide a user interface for this functionality, you must do a little extra work yourself.


Issues with Defining Collation at the Column Level

How often would you have a database that needs one sort order (for example, Latin1_General) and a column that needs a different one (for example Greek)? Sometimes this may be crucial, but in most cases if the data in your database does not use a single collation, then it is probably multilingual data that may need to be sorted according to more than one collation. Being able to define multiple collations, each of which can be indexed, allows you to access the Greek data by specifying the Greek collation and to have this query be an indexed search.


LCIDs and Collations

Windows uses the locale ID (LCID) to define sorts. If you are doing work to format your results, you will probably already have the LCID on hand (or you can use the default LCID by specifying 1024 or by using Microsoft Visual Basic formatting functions to do your work). In fact, if you are doing this work in a Web-based ASP application, you can use the SetLocale function in Microsoft Visual Basic Scripting Edition (VBScript) to change the formatting to use the date/time, number, and currency formatting preferences of any locale. Unfortunately, there is no way to map the two: You can get an LCID from a collation, but you cannot get a collation from an LCID due to many to 1 mapping from LCID to collation.

Why is this inconvenient? Well, imagine a case where you have a multilingual Web site, with people visiting from different countries and examining product information. You may already be mapping their browsers' HTTP_ACCEPT_LANGUAGE variables to LCIDs for formatting date and currency values using the Session.LCID property, and you decide that sorting using their locale is a sensible option for the sake of usability.


ISO Strings and Collations

You can obtain the HTTP_ACCEPT_LANGUAGE variable in VBScript with a script such as

Dim stLang
stLang = Request.ServerVariables("HTTP_ACCEPT_LANGUAGE")

Recognizing that this value is the only value many Web developers will have when it comes to locale information, the VBScript SetLocale function was designed to not only take LCID values but to accept this value directly. This means that you do not have to go through the intermediate step of mapping the value to an LCID. Because SQL Server 2000 does not accept a string such as "en-us" (English-United States) and properly map that to a Latin1_ General collation or a string such as "vi" (Vietnamese) and map that to a Vietnamese collation, you have to map all of them yourself.

How are custom collations defined? One commonly asked question that many developers have after seeing myriad collation options is how can they define their own. The answer is they cannot. Unless a collation is added to Windows 2000, it cannot be added to SQL Server 2000. That's because a collation is literally designed to define the method for sorting every defined character in the Unicode standard, and there is no user interface designed to allow for creating such a thing. (Note: All the new SQL Server collations are derived from information in Windows, which is why they are referred to as Windows collations.)


Multilingual Data in the User Interface

Although SQL Server is first and foremost a server, it has many administration and management tools. These tools have been updated in SQL Server 2000 to support multilingual data as needed.

SQL Server Enterprise Manager does a fairly good job of handling table names that are on the current default code page or server code page. It can also take advantage of some of the font-linking technologies in Windows to "borrow" characters from other fonts when needed. But font linking cannot do 100 percent of the job. There are many languages (such as Armenian, Sylfaen, Georgian, and Hindi) about which Windows has no advanced font linking information. Also, when a language uses characters that are not as commonly used within a script (such as Azeri–Cyrillic), most of the string will be displayed, but a few characters might not be.

There are three interesting things to note:

  1. When the name is not supported within the font and font linking cannot be done, you will not see characters are not corrupted; instead, they are replaced with boxes, clearly indicating that a character could not be displayed.

  2. Because the SQL Server 2000 client tools do not use the Uni-scribe technology for rendering complex scripts, unless you are using Windows 2000, the characters of bidirectional languages (such as Hebrew, Yiddish, Arabic, and Farsi) are displayed backwards. On Windows 2000 and BIDI-enabled platforms, however, these characters display properly. Other complex script-rendering issues, such as Thai word breaking, have the same limitations (although not on Windows 2000).

  3. The "base" font used by SQL Server Enterprise Manager is the one defined in the desktop display settings of the computer and cannot be overridden.


Multilingual Information in the Grid and SQL Panes of SQL Query Analyzer

In the SQL Server Enterprise Manager, you cannot make font changes; however, in the SQL Query Analyzer, in the Fonts tab in the Options dialog box, you can explicitly change the font for many parts of the user interface.

The reason for this may not be apparent at first, because font linking seems to allow most strings to be displayed. However, there is more to the proper display of a string than simply finding a font that can represent the characters. Often, the font choice can make a difference—therefore, this functionality can be important for properly representing multilingual data. In cases in which font linking does not seem to work, this feature enables the string to be displayed instead of boxes.


Format Issues in the Query Designer

In the Query Designer, for the most part, you can enter information in the grid pane that matches the default regional settings of the computer, or you can explicitly use the CONVERT function to cause a string in an arbitrary format to be handled.

There are a few design limitations with regard to this method of using the regional settings that you should be aware of:

  • Long data formats are not supported.

  • Currency symbols should not be entered in the grid pane, although the U.S. dollar sign ($) can optionally be used. Either way, the currency symbol retrieved from the regional settings will be used in the Results pane.

  • Unary minus always appears on the left side without parentheses, regardless of the regional settings. Thus, –1 should be represented as –1 instead of 1– or (1) or any other valid variation that may be specified in the Regional Options dialog box.

These limitations are necessary to allow a certain amount of worldwide support in the Query Designer, and are not actually something that will block most efforts to use locale-specific data.

Note that any information entered in the Grid pane will be translated to a locale-independent format in the SQL pane, thus "03.09.65" on a Standard German computer will be translated to {ts ' 1965-09-03 00:00:00 } . All data entered directly into the SQL pane should be in this format or, otherwise, include an explicit CONVERT call.


Sort Order

The sorting of data displayed in the Results pane is not influenced by the Regional Settings; instead, the collation rules control how an ORDER BY clause is interpreted.


Double-Byte (DBCS) Characters

You can enter DBCS characters for literals and database object names, aliases, parameter names, and parameter marker characters. However, you cannot use DBCS characters for SQL language elements such as function names or SQL keywords. Thus, you should use the keyword SELECT instead of the Japanese Fullwidth "SELECT".


Getting to SQL Server Data (Data Access Methods)

How you access data in SQL Server can be crucial. There are many different data access methods, and the rules for how each handles multilingual text can be very important. Descriptions of several of these data access methods follow.

OLE DB

OLE DB is the central component of the Microsoft Data Access Components (MDAC), which is MDAC version 2.1 in SQL Server 7.0 and MDAC version 2.6 in SQL Server 2000. OLE DB is based on COM; thus all the strings are Unicode BSTRs (UTF-16 on Windows 2000 and UCS-2 on all other operating systems). For SQL Server, the provider is the Microsoft OLE DB Provider for SQL Server (SQLOLEDB). Data is converted to Unicode as needed, using the collation of the actual data. The most optimized case would require you to keep the data as Unicode throughout the process.

ADO

Microsoft ActiveX Data Objects is a Visual Basic and scripting friendly interface that acts as a wrapper around OLE DB. It is also a COM component and thus has the same support for Unicode. There is no way to decouple ADO and OLE DB in a way that would ever allow conversions to happen between the two, so when problems do exist, they will always be at the OLE DB layer.

ODBC

Whether ODBC is a Unicode layer depends on the version of ODBC being used.

DB-Library

There is no later Unicode version of DB-Library.

SQL-DMO

SQL Distributed Management Objects (SQL-DMO) is a COM layer that encapsulates SQL Server 2000 database and replication management. Because it is COM, the same rules that applied to ADO and OLE DB apply to SQL-DMO. SQL-DMO also has properties that can be used for features mentioned earlier, such as the Collation property on the SQLServer2, Database2, Column2, SystemDateType2, and UserDefinedDataType2 objects.


Multilingual Transact-SQL

When you send an SQL statement to the server that contains multilingual data, the three primary issues that will affect whether the data makes it to the server properly are the encoding of the SQL statement itself, the encoding of string literals inside the statement, and copying data between different collations. When bulk copying data using native or character format, bcp by default converts character data to OEM code page characters when exporting data from an instance of Microsoft SQL Server and ANSI/Microsoft Windows code page characters when importing data into an instance of SQL Server. This can cause the loss of extended or DBCS characters during the conversion between OEM and ANSI code pages.

To prevent the loss of extended or DBCS characters, bcp can create data files using Unicode native data format (-N); Unicode character data format (-w), and a specific code page (-C). Unicode native format and Unicode character format convert character data to Unicode during the bulk copy, resulting in no loss of extended characters.

Using the -C (code page) switch, the bcp utility can create or read data files using the code page specified by the user. For example, to bulk copy the authors2 table in the pubs database to the Authors.txt data file using code page 850, execute from the command prompt:

bcp pubs..authors2 out authors.txt -c -C850 -Sservername -Usa -Ppassword

Alternatively, using the CODEPAGE clause, the BULK INSERT statement can read data files using the code page specified by the user. For example, to bulk copy the Authors.txt data file into the authors2 table in the pubs database using code page 850, execute from a query tool such as SQL Query Analyzer:

BULK INSERT pubs..authors2 FROM 'c:\authors.txt'
WITH (
   CODEPAGE = 850
)


Summary

In this chapter we delved into the new collations that SQL Server 2000 provides. The biggest advantage is that collations can now be specified at different levels; SQL Server does not need to be reinstalled for making changes to the sort order or the code page. More information on Unicode can be found at http://www.unicode.org.


Part II: Advanced DBA Topics

In this part we discuss some advanced topics for the DBAs including the use of SQL-DMO to create sample applications to aid in administration and to restrict the functionality provided by the Enterprise Manager. We discuss MSDE 2000 and what it can be used for. We then discuss the various database consistency checks provided by SQL Server 2000 (the documented as well as the undocumented ones and how they can be used), and in the last chapter of this part we discuss how to debug the performance problems with the databases.


Chapter 20. SQL-Distributed Management Objects

When a colleague of mine first told me about SQL Distributed Management Objects (SQL-DMO), I thought, "Who needs another way to manage databases when I've got T-SQL and Enterprise Manager?" But when I needed to manage many servers hosting large, complex databases, I discovered the power of using SQL-DMO objects embedded in Windows Script Host scripts.

In this chapter we take a look at some real-world scripts that use SQL-DMO objects and that can help you manage an enterprise environment where T-SQL scripts and Enterprise Manager aren't enough. You can use Enterprise Manager to manually browse through your tables and other entities, physically looking for a particular name. However, when you need to search several databases, each of which might have hundreds of tables with thousands of columns, this approach becomes tedious and time-consuming. Another option is to execute SQL queries against system tables such as syscolumns, specifying your search target in the WHERE clause. You might also use ANSI-standard SQL information_schema views to enable easy querying of database object meta data. Querying these views is preferable to querying system tables for a couple of reasons. First, you'd have to query the sysobjects and syscolumns tables to get the information that the table query returns. And second, Microsoft doesn't guarantee that the system tables will remain the same between versions and service packs.

Although SQL queries work, to get the information you need, you must understand the internal references that SQL Server uses to keep track of its entities and the relationships between them. This approach trades the logistical nightmare of a manual search for the headache of getting deep down into the database wires. Although the information_schema views don't require you to understand the internal references in the system tables, SQL Server's SQL Distributed Management Objects (SQL-DMO) provides a balance between ease of use and powerful object-manipulation capabilities.


What Is SQL-DMO?

SQL-DMO is a COM-compliant type library that lets you programmatically manipulate native SQL Server objects. In fact, Microsoft used SQL-DMO to build the Enterprise Manager. You can leverage SQL-DMO's capabilities with any COM-compliant development language, such as Visual Basic (VB), C++, or VBScript. This chapter uses VB and SQL-DMO to build an advanced search utility that goes beyond the Enterprise Manager's capabilities to locate SQL Server entities by name (or partial name). SQL-DMO is intended for any Automation or COM application that must incorporate SQL Server administration, for example, applications that encapsulate SQL Server as their data store and want to shield users from as much SQL Server administration as possible; applications that have specialized administrative logic incorporated the application itself; and applications that want to integrate SQL Server administrative tasks in their own user interface. SQL-DMO automates repetitive or commonly performed SQL Server administrative tasks, SQL Server object creation and administration, creation and administration of SQL Server Agent jobs, alerts, and operators, and SQL Server replication installation and configuration.


SQL-DMO Object Model

Figure 20.1 shows a simplified version of the SQL-DMO object model, which still consists of many objects and collections representing both administrative activities and database structures. These objects expose properties, methods, and events for your programmatic use, as do all COM components. For example, to programmatically execute an SQL Server backup, you'd use an instance of the SQL-DMO Backup object. You'd set various properties, such as Database and TruncateLog, to specify the sort of backup you want to perform. Then you'd invoke the SQLBackup method to actually back up the database. While the backup is running, your program might use the PercentComplete event to display a dynamic status indicator. You can invoke most of SQL Server's administrative functions in this way, including starting and stopping servers, executing jobs, and configuring replication.

Figure 20.1. SQL-DMO object model

Manipulating or examining the structure of databases follows a similar process; however, the concept of collections becomes important. You might have multiple registered servers on a computer, each containing multiple databases, each of which contains multiple tables, and so on. Collections gather these entities together into an abstract construct that you can enumerate programmatically.

Dim dmoApp As SQLDMO.Application
Dim dmoServerGroup As SQLDMO.ServerGroup
Dim dmoServer As SQLDMO.RegisteredServer

Set dmoApp = New SQLDMO.Application
For Each dmoServerGroup In dmoApp.ServerGroups
     For Each dmoServer In dmoServerGroup.RegisteredServers
          'Add the server name to a list box
          lstServers.AddItem dmoServer.Name
     Next
Next

The preceding code shows how to use SQL-DMO to find all the registered SQL Server systems on your computer. First, the code instantiates the SQL-DMO Application object, whose ServerGroups property serves as a pointer to a collection of ServerGroup objects. These objects represent groups of SQL Server systems that you've registered on your computer. Typically, you'll have only one server group, but the collection concept allows any number. Then, for each server group the code finds, the RegisteredServers property points to a collection of RegisteredServer objects. Finally, the code adds each registered server's name to a list box on a VB form. The process is quite straightforward.


ExecuteImmediate Method

ExecuteImmediate gives you the ability to execute T-SQL or stored procedures from within your DMO script. If you use ExecuteImmediate as a method of a database object, you get the same effect as if you had executed a Use Database in QueryAnalyzer. You can also use ExecuteImmediate as a method of the server object, in which case the database is always the master. Assuming you've already established a connection to your server, the following sample code will update statistics on all objects in all databases.

Dim oServer        'As SQLDMO.SQLServer
Dim oDatabase      'As SQLDMO.Database
Set oServer = CreateObject("SQLDmo.SqlServer")
oServer.LoginSecure = True
oServer.Connect "(local)"

For Each oDatabase In oServer.Databases
    '2=SQLDMOExec_ContinueOnError
    oDatabase.ExecuteImmediate "sp_updatestats", 2
Next

'clean up
oServer.DisConnect
Set oServer = Nothing


Script Method

Think of the scripting options available in the Enterprise Manager. In DMO, you can reproduce them all, plus some! In this example I'm creating one script per database containing all its views. I'm using the AppendToFile flag so that each time I script an object, it doesn't overwrite the previous script. The PrimaryObject flag is the one that tells DMO to generate the DDL for the object.

This example also makes use of the CommandShellImmediate method—which directly corresponds to xp_cmdshell. Take a look also at the nested loops, the outer one for the databases, the inner for the views—objects make this kind of looping incredibly easy.

Dim oServer      'As SQLDMO.SQLServer
Dim oDatabase    'As SQLDMO.Database
Dim oView        'As SQLDMO.View

Set oServer = CreateObject("SQLDmo.SqlServer")

oServer.LoginSecure = True
oServer.Connect "(local)"

'this deletes previous versions of scripts - use with care!
oServer.CommandShellImmediate "Delete C:\DMO_Views*.sql"

'loop through each view in each database, creating one script per database
'to create all of the views
For Each oDatabase In oServer.Databases
    For Each oView In oDatabase.Views
        'SQLDMOScript_AppendToFile=8192
        'SQLDMOScript_ObjectPermissions=2
        'SQLDMOScript_ToFileOnly=64
        'SQLDMOScript_PrimaryObject=4
        oView.Script 8192 + 2 + 64 + 4, "C:\DMO_Views_" & oDatabase.Name & ".sql"
    Next
Next

'clean up
oServer.DisConnect
Set oServer = Nothing

Msgbox "Done."


SQL-DMO Code for Executing a Job

Sub RunAJob(ServerName As String, JobName As String)
Dim oServer As SQLDMO.SQLServer
Dim oJob As SQLDMO.Job
On Error GoTo Handler

'simple err checking
If ServerName = "' Or JobName = "' Then
MsgBox "You MUST provide the server name a  job name.", vbInformation + vbOKOnly, "Error"
Exit Sub
End If

'open connection to server using a trusted connection
Set oServer = New SQLDMO.SQLServer
With oServer
.LoginSecure = True
.Connect ServerName
End With

'run the job
Set oJob = oServer.JobServer.Jobs(JobName)
If oJob.CurrentRunStatus = SQLDMOJobExecution_Idle Then
'show a little info just to look at the job properties
MsgBox "Job was last run at " & oJob.LastRunDate & " " & oJob.LastRunTime & "."
'Im using invoke here, but if you don't want to start at the default step you
'should use the Start method
oServer.JobServer.Jobs(JobName).Invoke
Else
MsgBox "Job cannot be executed"
End If
Set oJob = Nothing

'standard clean up
Cleanup:
On Error Resume Next
oServer.DisConnect
Set oServer = Nothing
Exit Sub
Handler:
MsgBox Err.Number & "-" & Err.Description, vbCritical + vbOKOnly, "Error"
GoTo Cleanup
End Sub


SQL-DMO Code to Play with Triggers

Sample stored procedures are given in Chapter 27 for disabling and enabling triggers using Transact-SQL, but disabling and enabling triggers can also be achieved by using SQL-DMO.

Enabling the Triggers

Dim oServer
Dim oTable
Dim oTrigger
dim DBName

'change this to your database name
DBName='PUBS'

Set oServer = createobject("SQLDMO.SQLServer")

oServer.LoginSecure = True
oServer.Connect

For Each oTable In oServer.Databases(DBName).Tables
    For Each oTrigger In oTable.Triggers
        oTrigger.enabled=true
    Next
Next

oServer.DisConnect
Set oServer = Nothing

Disabling theTriggers

Dim oServer
Dim oTable
Dim oTrigger
dim DBName

'change this to your database name
DBName='PUBS'

Set oServer = createobject("SQLDMO.SQLServer")
oServer.LoginSecure = True
oServer.Connect

For Each oTable In oServer.Databases(DBName).Tables
    For Each oTrigger In oTable.Triggers
        oTrigger.enabled=false
    Next
Next

oServer.DisConnect
Set oServer = Nothing

There is a sample VBScript in Chapter 28 which can be used for backing up the databases. It uses SQL-DMO to loop through the database and back them up.


Summary

In this chapter, we explored some of the scenarios for which you can use SQL-DMO. We went over the object model of SQL-DMO and looked at some code snippets for doing some of the common tasks. Knowledge of SQL-DMO proves to be handy in situations where you just want to use MSDE as your database engine for lesser loads. Since MSDE does not ship with the client tools, you can use SQL-DMO to build your own enterprise manager.


Chapter 21. Microsoft Desktop Engine

The SQL Server 2000 Desktop Engine is the successor to Microsoft Data Engine (MSDE) 1.0, which was based on SQL Server version 7.0. As such, the SQL Server 2000 Desktop Engine is often referred to as MSDE 2000. The SQL Server 2000 Desktop Engine is not related to SQL Server 7.0 Desktop Edition. Probably the most significant characteristic of the Desktop Engine is that it is a redistributable version of the SQL Server relational database engine. The Desktop Engine is an ideal embedded or offline data store, because it is easy to install and has the smallest footprint of any edition of SQL Server 2000.

The SQL Server 2000 Desktop Engine does not include graphical management tools; the application distributing the engine is usually coded to perform any needed database administration. You can manage instances of the Desktop Engine from the SQL Server 2000 graphical tools if installed with another edition of SQL Server. Other items not included in the Desktop Engine include analysis capabilities (such as OLAP, data mining, and data warehousing features) and SQL Server Books Online. The Desktop Engine also limits database size to 2 GB and employs the same concurrent workload throttle described earlier for the Personal Edition.

Like all of the editions of SQL Server 2000, the Desktop Engine supports multiple instance installations, lessening the likelihood that an instance installed by an application of one vendor will conflict with subsequent MSDE-based applications installed by applications of other vendors. Additionally, all of the APIs and most of the functionality of the other editions of SQL Server 2000 are supported. The Desktop Engine includes the SQLServerAgent service for managing scheduled tasks. Although the Desktop Engine does not include the management tools or wizards, applications can fully administer an instance of the Desktop Engine using the SQL Server administration APIs, such as SQL-DMO, the DTS and Replication programming objects, or the general database APIs (such as ADO, OLE DB, and ODBC). Applications can use the general database APIs to access data in theDesktop Engine, and the Desktop Engine can participate alongside other editions of SQL Server 2000 in DTS transformations and replication plans (except operating as a transactional replication publisher).


Portable and Compatible

MSDE 2000 is a royalty-free, redistributable database engine that is fully compatible with SQL Server. MSDE 2000 is designed to run as an embedded database for custom applications that require a local database engine on Microsoft Windows 98, Windows Millennium Edition, Windows NT Workstation 4.0 (with Service Pack 5 or higher), and Windows 2000 Professional.

MSDE 2000 is an attractive alternative to the Microsoft Jet database and is designed primarily to provide a low-cost option for developers who need a database server that can be easily distributed and installed with a value-added business solution. Because it is fully compatible with other editions of SQL Server, developers can easily target both SQL Server and MSDE 2000 with the same core code base. This provides a seamless upgrade path from MSDE 2000 to SQL Server if the application grows beyond the storage and scalability limits of MSDE 2000.

Minimum Requirements

To run MSDE 2000, you must meet the following minimum system requirements:

  • PC with an Intel or compatible Pentium 166 MHz or higher processor

  • Windows 98, Windows Millennium Edition, Windows NT Workstation 4.0 with Service Pack 5 or later, Windows NT Server 4.0 with Service Pack 5 or later, Windows NT 4.0 Server Enterprise Edition with Service Pack 5 or later, Windows 2000 Professional, Windows 2000 Server, Windows 2000 Advanced Server, or Windows 2000 Datacenter Server operating system

  • Minimum of 32 MB of RAM. (64 MB or more is recommended. Additional memory may be required, depending on operating system requirements.)

  • 44 MB of available hard disk space


Features and Limitations

Multi-instance support

MSDE 2000 supports up to 16 database server instances on a single computer.

Osql.exe

The osql utility is a Microsoft Win32 command prompt utility for ad hoc interactive execution of Transact-SQL statements and scripts. With this utility, you can interactively enter Transact-SQL statements in a manner similar to working on the command prompt. The results are displayed in the command prompt window.

Replication

MSDE 2000 supports merge replication both as a publisher and as a subscriber, allowing you to keep copies of the same data on multiple sites, sometimes hundreds of sites. MSDE 2000 also supports other forms of replication but only in a limited capacity. MSDE 2000 can participate in transactional replication and snapshot replication in a subscriber capacity only.

Performance

MSDE 2000 is a local data engine that can be shared. MSDE 2000 has a managed concurrency workload governor that limits shared use to eight or fewer concurrent batch workloads for optimal performance. Commands and log entries are available to monitor instances where more than five workloads are executed concurrently, which can cause slower performance even on well-tuned systems. As more batch workloads are submitted beyond the five-workload limit, the concurrency governor continues to slow down the system. These workloads are not dropped or lost; they are still processed, but in an increasingly degraded performance mode. If your solution must support more than five concurrent workloads, it is highly recommended that you migrate to SQL Server 2000 Standard Edition or SQL Server 2000 Enterprise Edition for optimal performance at this higher level of scalability.

Maximum database size

MSDE 2000 supports up to 2 GB per database. This limitation is per database, not per server. A single computer can support multiple MSDE 2000 instances, each having databases up to 2 GB in capacity.

Data transformation services

MSDE 2000 is capable of running data transformation services (DTS) packages. However, it cannot design DTS packages because it is not equipped with a DTS designer.

Remote administration

It is possible to administer MSDE 2000 locally and remotely. However, MSDE 2000 cannot be remotely administered in a multiserver environment where transactions occur across servers.


MSDE Install

The primary way to install the SQL Server 2000 Desktop Engine is to use the Windows Installer service in Microsoft Windows 2000 and available for Windows NT 4.0, Windows 95, and Windows 98. You can use the following methods to install the Desktop Engine using Windows Installer.

  • Microsoft SQL Server 2000 provides a set of Windows Installer merge modules that can be used to install an instance of the Desktop Engine. These can be integrated into a standard Windows Installer setup procedure.

  • An application setup program that calls Windows Installer directly can install an instance of the Desktop Engine.

The Desktop Engine also produces a number of return codes that indicate the success or failure of an attempt to install an instance of the Desktop Engine using Windows Installer.

The Microsoft SQL Server 2000 Desktop Engine includes a Setup.exe file that an application setup process can use to install the Desktop Engine. You can define settings for customizing the way Setup.exe installs the Desktop Engine. These settings can be specified through either command prompt switches or an .ini file.

Syntax

setup [/?]
   [
      [   /i package_file
          [ /settings ini_file ]
          |   [   [ TARGETDIR="executable_folder_path"]
                   [ DATADIR="data_folder_path" ]
                   [ INSTANCENAME="instance_name" ]
                   [ COLLATION="collation_name" ]
                   [ CALLBACK=Dllname!CallbackFunctionName ]
                   [ USEDEFAULTSAPWD | SAPASSWORD="sa_password" ]
              ]
      ]
      | [/x package_file ]
          ]
          [ /L*v [filename] ]
          [ /qn | /qb ]

Arguments

/? displays a syntax summary of the setup switches. setup displays more switches than it accepts; use only the switches documented in this topic.

/i package_file specifies the name of the Windows Installer installation package file (an .msi file) to be used to install an instance of the SQL Server 2000 Desktop Engine. Place the .msi file in the same folder as Setup.exe. If /i is not specified, copy all 16 of the .msi files from the \Sqlmsde\Setup folder on the SQL Server 2000 compact disc to the same folder as Setup.exe.

settings ini_file_name specifies the name of an .ini file containing settings for TARGETDIR, DATADIR, INSTANCENAME, COLLATION, USEDEFAULTSAPWD, and SAPASSWORD. If /settings is specified, these options should be set in the .ini file, not in the command prompt switches. Place the .ini file in the same folder as Setup.exe.

TARGETDIR="executable_folder_path" specifies the folder where the Desktop Engine executable files are installed. The default for executable_folder_path is C:\Program Files\Microsoft SQL Server.

DATADIR="data_folder_path" specifies the folder where the SQL Server system databases are built. The default is the same folder used for the Desktop Engine executable files.

INSTANCENAME="instance_name" specifies the name for the instance. If no instance name is specified, the instance is installed as a default instance.

COLLATION="collation_name" specifies the SQL Server collation that will be used as the default collation for this instance of the Desktop Engine.

CALLBACK=Dllname!CallbackFunctionName specifies the name of the DLL containing the Desktop Engine Windows Installer callback function and the name of the callback function.

USEDEFAULTSAPWD specifies that setup assign the default password (NULL) to the sa login.

SAPASSWORD="sa_password" specifies that Setup assign a password to the sa login. When /i is specified, you must specify either /USEDEFAULTSAPWD or /SAPASSWORD but not both.

/x package_name specifies the name of the Windows Installer installation package file (an .msi file) to use when uninstalling an instance of SQL Server 2000 Desktop Engine. You must specify the name of the same installation package file that was used to install the instance of the Desktop Engine. Place the .msi file in the same folder as Setup.exe.

/L [image] v [filename] specifies that a verbose log be created. If filename is specified, the log is stored in the file specified.

/qn specifies that Setup run with no user interface.

/qb specifies that Setup show only the basic user interface. Only dialog boxes displaying progress information are displayed. Other dialog boxes, such as the dialog box that asks users if they want to restart at the end of the setup process, are not displayed.

SQL Server 2000 Desktop Engine Setup is a tool that applications can use to install the SQL Server 2000 Desktop Engine as a part of the typical installation process. It is not intended as a tool for installing the database engine from the SQL Server 2000 compact disc. To install SQL Server 2000 from a compact disc, use the autorun.exe component in the root folder of the SQL Server 2000 compact disc.

Applications that do not use Windows Installer as their setup mechanism can use the SQL Server 2000 Desktop Engine Setup. Applications that use Windows Installer should instead use the SQL Server 2000 Desktop Engine Merge Modules.

To run SQL Server 2000 Desktop Engine Setup, the application setup process must copy these files to the same folder:

  • Setup.exe from the \Sqlmsde folder on the SQL Server 2000 compact disc

  • Sqlrun.cab from the \Sqlmsde\Setup folder on the SQL Server 2000 compact disc

  • Either all of the SqlrunXX.msi installation package files from the \Sqlmsde\Setup folder on the SQL Server 2000 compact disc or a custom installation package file

  • Optionally, an .ini file with settings for the TARGETDIR, DATADIR, INSTANCE NAME, COLLATION, USEDEFAULTSAPWD, and SAPASSWORD options.

The application then executes Setup.exe at the point in the installation process where the Desktop Engine should be installed. The setup process of an application typically uses one of two mechanisms to execute the Desktop Engine Setup:

  1. Execute Setup.exe from a batch command file or from the command prompt. When using this method, setup switches are specified in the same way they would be specified for any other command prompt utility.

  2. Execute Setup.exe from within the application setup executable file using a method similar to the Win32 CreateProcess() function. In this case, setup switches are specified in a character string passed in using the lpCommandLine paramter.

If neither /qn nor /qb is specified, Setup.exe displays all user interface dialog boxes.

Whenever an application needs to uninstall an instance of the Desktop Engine, it must call the SQL Server 2000 Desktop Engine Setup using the /x switch to specify the same .msi file that was used to install that instance. For example, to uninstall a named instance installed with a custom installation package file named MyCustom.msi, execute:

Setup /x MyCustom.msi

Note

When the MSDE install goes through, the default authentication mode is the Windows authentication mode, so if you want to change this, you can change the registry setting:

HKEY_LOCAL_MACHINE\Software\Microsoft\MSSQLServer\MSSQLServer\LoginMode

to a value of 2, which puts the authentication to the mixed mode. After doing this, you will need to restart the SQL Server services for the change to take place. Alternatively, there is an undocumented option during the install called SECURITYMODE, which sets the authentication level to mixed mode. To set up mixed-mode authentication you can use

SECURITYMODE=SQL parameter.
SAPASSWORD/USEDEFAULTSAPWD are ignored.


Scenarios for Using MSDE

MSDE has the smallest footprint of all SQL Server engines. It has a limitation of 2 GB of data and a limitation five concurrent connections before the performance degrades. One of the ideal scenarios for MSDE applications is when you need to have a repository at the Web server for a small number of tables. These tables will most probably be read-only tables, which you can even cache into memory using either DBCC PINTABLE or sp_tableoption. Since your application code will not have to go to the database server for the lookup data, that will prevent a network round trip and hence will be very efficient in cases where the LAN speed is a consideration. MSDE can also serve as a repository.


Developing Applications with MSDE

Since MSDE does not come with client tools, you can develop your own set of client tools using SQL-DMO. You can use the sample application we created in Chapter 20 using SQL-DMO for administering your MSDE databases. With MSDE 2000, application developers are given the ability to install a redistributable relational database engine fully compatible with SQL Server 2000 as part of the custom application setup process. Application solutions using MSDE 2000 can store data in a desktop engine whose configuration and resources are managed dynamically, removing the need for extra graphical tools and administrative utilities. This makes MSDE 2000 the ideal database to embed into desktop and laptop applications.

Application developers can build the MSDE 2000 installation into their application setup. After the database engine is installed, developers can use standard SQL Server APIs to build and manage database objects.

Sample Application

The CD-ROM with this book includes a sample application that you can use to do the following:

  • Check for an existing version of SQL on the box

  • Install MSDE (expects the CD to be in the drive)

  • Detach and attach the specified database

The entire code base for the application is in the MSDE folder on the CD-ROM. It is written in VB and has been commented appropriately.

MSDE Service Pack1

MSDE (Desktop Engine) 2000 Service Pack 1 was released with SQL Server 2000 Service Pack1 (it is discussed in more detail in Chapter 23). The way to install the MSDE service Pack is to run it from the command line rather than running the Setup.exe directly. The command line is: Setup.exe /p SQLRUN


Summary

In this chapter we took a look at the feature sets of MSDE, what it can be used for, and its advantages and limitations.


Chapter 22. Administration

In this chapter we take a look at some advanced administrative topics essential for any database administrator to know for successful implementation and maintenance.


Database Consistency Checks (DBCCs)

The Transact-SQL programming language provides DBCC statements that act as database console commands for Microsoft SQL Server 2000. These statements check the physical and logical consistency of a database. Many DBCC statements can fix detected problems.

Database console command statements are grouped into the categories shown in Table 22.1. The DBCC statements of SQL Server 2000 take input parameters and return values. All DBCC statement parameters can accept both Unicode and DBCS literals. Following are the key DBCC commands that are documented.

DBCC CHECKDB command checks all the tables in the specified database. It also runs the DBCC CHECKALLOC command (checks the database for corresponding space allocations) and the DBCC CHECKTABLE command (checks structures for individual tables), and it supersedes the DBCC TEXTALLOC/TEXTALL command.

Three repair options that can be used with the DBCC CHECKDB command in the event the DBCC CHECKDB command displays database errors:

  • REPAIR_FAST performs a minor, quick repair on the database without the risk of data loss should a table be corrupt.

  • REPAIR_REBUILD performs a more extensive repair procedure than REPAIR_FAST. It includes rebuilding the indexes and can also be done without the risk of data loss.

  • REPAIR_ALLOW_DATA_LOSS performs the most comprehensive repair procedure. This option may result in data loss and may require a restore of the database from a backup.

Table 22.1. Database console command statements
Statement category | Function
Maintenance statements | Maintenance tasks on a database, index, or filegroup
Miscellaneous statements | Miscellaneous tasks such as enabling row-level locking or removing a dynamic-link library (DLL) from memory
Status statements | Status checks
Validation statements | Validation operations on a database, table, index, catalog, filegroup, system tables, or allocation of database pages

DBCC CHECKCATALOG command checks the system tables and confirms that every object contains at least one column and that every column contains a valid data type.

DBCC CHECKIDENT checks the current identity value for the specified table and, if needed, corrects the identity value.

DBCC DBREINDEX rebuilds one or more indexes for a table in the specified database. The syntax is as follows.

DBCC DBREINDEX
    (    [ 'database.owner.table_name'
            [ , index_name
                [ , fillfactor ]
            ]
        ]
    )    [ WITH NO_INFOMSGS ]

DBCC PINTABLE marks a table to be pinned, which means that Microsoft SQL Server does not fiush the pages for the table from memory. Its syntax is

DBCC PINTABLE ( database_id , table_id )

DBCC INDEXDEFRAG defragments clustered and secondary indexes of the specified table or view. Its syntax:

DBCC INDEXDEFRAG
    ( { database_name | database_id | 0 }
        , { table_name | table_id | 'view_name' | view_id }
        , { index_name | index_id }
    )    [ WITH NO_INFOMSGS ]

Following are the key DBCC commands that are not documented.

DBCC BUFFER can be used to print buffer headers and pages from the buffer cache.

Syntax

Dbcc buffer ([dbid|dbname] [,objid|objname] [,nbufs], [printopt])

where

dbid|dbname— database id|database name.

objid|objname— object id|object name

nbufs— number of buffers to examine

printopt— print option

0— print out only the buffer header and page header (default)

1— print out each row separately and the offset table

2— print out each row as a whole and the offset table

Example

DBCC Traceon (3604)

dbcc buffer (master, 'sysindexes')

DBCC BYTES can be used to dump bytes out of a specific address.

Syntax

Dbcc bytes (stataddress, length)

where

startaddress— starting address to dump

length— number of bytes to dump

Example

DBCC Traceon (3604)

Dbcc bytes (10000000, 100)

DBCC DBINFO prints DBINFO structure for specified database.

Syntax

Dbcc dbinfo [(dbname)]

where

dbname— is the database name

Example

DBCC Traceon (3604)

Dbcc dbinfo(master)

DBCC DBTABLE prints out the contents of the DBTABLE structure.

Syntax

Dbcc dbtable ({dbid|dbname})

where

dbid|dbname— database name or database ID

Example

DBCC Traceon (3604)

Dbcc dbtable (master)

The DBTABLE structure has an output parameter called dbt_open. This parameter keeps track of how many users are in the database.

DBCC DES prints the contents of the specified DES (descriptor).

Syntax

Dbcc des [( [dbid|dbname] [,objid|objname] )]

where

dbid|dbname— database id|database name

objid|objname— object id|object name

Example

DBCC Traceon (3604)

Dbcc des

DBCC IND shows all pages in use by indexes of the specified table.

Syntax

dbcc ind( dbid|dbname, objid|objname, printopt = {-2|-1|0|1|2|3})

where

dbid|dbname— database id|database name

objid|objname— object id|object name

printopt— print option

There is change in this command in comparison with SQL Server 2000 one: You should specify printopt parameter (this parameter is not optional now)

Example

DBCC Traceon (3604)

DBCC IND (master, sysobjects, 0)

DBCC log is used to view the transactional log for the specified database.

Syntax

DBCC log ( {dbid|dbname}, [, type={-1|0|1|2|3|4}] )

where

Dbid or dbname—Enter either the dbid or the name of the database in question.

Type—Type of output:

0—Minimum information (operation, context, transaction ID).

1—More information (plus fiags, tags, row length, description).

2—Very detailed information (plus object name, index name, page ID, slot ID).

3—Full information about each operation.

4—Full information about each operation plus hexadecimal dump of the current transaction log's row.

–1—Full information about each operation plus hexadecimal dump of the current transaction log's row, plus Checkpoint Begin, DB Version, Max XDESID by

default type = 0.

To view the transaction log for the master database, you can run the following command:

DBCC log(master)

DBCC PAGE command is used to view the data page structure.

Syntax

DBCC PAGE ({dbid|dbname}, pagenum [,print option] [,cache] [,logical])

Parameters

Dbid or dbname—Enter either the dbid or the name of the database in question.

Pagenum—Enter the page number of the SQL Server page that is to be examined.

Print option (optional)—Can be either 0, 1, or 2.

0 (default)—Causes DBCC PAGE to print out only the page header information.

1—Causes DBCC PAGE to print out the page header information, each row of information from the page, and the page's offset table. Each of the rows printed out will be separated from the others.

2—Same as option 1 except that the page rows are printed as a single block of information rather than as individual rows. The offset and header are also be displayed.

Cache (optional)—Allows either a 1 or a 0 to be entered.

0—Causes DBCC PAGE to retrieve the page number from disk rather than checking to see if it is in cache.

1 (default)—Takes the page from cache if it is in cache rather than getting it from disk only.

Logical (optional)—used if the page number that is to be retrieved is a virtual page rather than a logical page. It can be either 0 or 1.

0—If the page is to be a virtual page number.

1 (default)—If the page is the logical page number.

Example

DBCC Traceon (3604)

Dbcc page (master, 1, 1)

DBCC prtipage prints the page number pointed to by each row on the specified index page.

Syntax

DBCC prtipage( dbid, objid, indexid, indexpage)

where

dbid— database ID

objid— object ID

indexid— index ID

indexpage— logical page number of the index page to dump

Example

DBCC Traceon (3604)

DECLARE @dbid int, @objectid int
SELECT @dbid = DB_ID('master')
SELECT @objectid = object_id('sysobjects')
DBCC prtipage(@dbid,@objectid,1,0)

DBCC pss shows information about processes currently connected to the dataserver. Structure is the source of data contained in the sysprocesses table.

Syntax

DBCC pss( suid, spid, printopt = {1 | 0 })

where

suid— server user ID

spid— server process ID

printopt— print option (0 standard output, 1 all open DES's and current sequence tree)

Example

DBCC Traceon (3604)

dbcc pss

DBCC resource shows dataserver level RESOURCE, PERFMON and DS_CONFIG info. RESOURCE shows addresses of various data structures used by the server. PERFMON structure contains master..spt_monitor field info. DS_CONFIG structure contains master..syscurconfigs field info.

Syntax

Dbcc resource

Example

DBCC Traceon (3604)

Dbcc resource

DBCC TAB can be used to view the data pages structure. (In contrast to DBCC PAGE, this command returns information about all data pages for a viewed table, not just a few.

Syntax

Dbcc tab (dbid, objid)

where

dbid— database id

objid— table id

Example

DBCC Traceon (3604)

DECLARE @dbid int, @objectid int
SELECT @dbid = DB_ID('master')
SELECT @objectid = object_id('sysdatabases')
DBCC TAB (@dbid,@objectid)


Information Schema Views

Microsoft SQL Server 2000 ships with some information schema views that are very helpful for getting information about the meta data. Microsoft suggests using these views instead of querying the system tables—systypes, sysreferences, sysindexes, and so on—because the tables can change from release to release. I personally prefer to have a good knowledge of the system tables and occasionally use them for database administration purposes, but for the application it's good if your SQL queries are based off these views so you are sure that when you migrate from one version to the other of SQL Server, you are not going to be breaking something. All these information_schema views exist in SQL Server 2000 and detailed information on all of them is available in BOL, so I won't go into the details of what each view does. Here is the list of the views:

Information_Schema.Check_Constraints

Information_Schema.Column_Doman_Usage

Information_Schema.Column_Privleges

Information_Schema.Columns

Information_Schema.Constraint_Column_Usage

Information_Schema.Constraint_Table_Usage

Information_Schema.Domain_Constraints

Information_Schema.Domains

Information_Schema.Key_Column_Usage

Information_Schema.Parameters

Information_Schema.Referential_Constraints

Information_Schema.Routine_Columns

Information_Schema.Routines

Information_Schema.Schemata

Information_Schema.Table_Constraints

Information_Schema.Table_Constraints

Information_Schema.Table_Privileges

Information_Schema.Tables

Information_Schema.Views

Information_Schema.View_Column_Usage

Information_Schema.View_Table_Usage

Some examples:

  • For getting a list of constraints, column_names and their position in the constraint:

    SELECT
COLS.CONSTRAINT_NAME,COLS.COLUMN_NAME,COLS.ORDINAL_POSITION
FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE AS COLS
INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS CONS ON
COLS.CONSTRAINT_NAME = CONS.CONSTRAINT_NAME
WHERE COLS.CONSTRAINT_CATALOG = DB_NAME()
AND COLS.TABLE_NAME = 'TASK_DTL'
AND CONS.CONSTRAINT_TYPE = 'PRIMARY KEY'
ORDER BY COLS.CONSTRAINT_NAME, COLS.ORDINAL_POSITION

    You can replace the name of the table with whatever you want. You can also replace the constraint_type if you want to search for constraints other than the primary key.

    SELECT CONSTRAINT_NAME,
       COLUMN_NAME,
       ORDINAL_POSITION
FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE CONSTRAINT_CATALOG = DB_NAME()
AND TABLE_NAME = 'X'
ORDER BY CONSTRAINT_NAME, ORDINAL_POSITION

    Replace X with the name of the table and you will get the name of the constraints defined on the table (primary key as well as the foreign key constraints), the names of the columns involved, and the position of the columns in the constraint for example, which column is number 1 in the case of a covered primary key.

  • For a list of tables, their columns, data type for the columns, NULL/Not NULL criteria:

    SELECT A.TABLE_NAME, B.COLUMN_NAME, B.DATA_TYPE, B.IS_NULLABLE
FROM INFORMATION_SCHEMA.TABLES A, INFORMATION_SCHEMA.COLUMNS B
WHERE A.TABLE_NAME = B.TABLE_NAME
ORDER BY A.TABLE_NAME


Orphaned Sessions

An orphaned session is a session that remains open on the server side after the client has disconnected. Do not confuse orphaned sessions with orphaned users. Orphaned users are created when a database is backed up and restored to another system on which no corresponding user account is configured. Orphaned sessions occur when the client is unable to free network connections it is holding when it terminates. If the client terminates cleanly, Windows closes the connection and notifies SQL Server. If SQL Server is processing a client command, it detects the closed connection when it ends the session. Client applications that crash or have their processes killed (for example, from Task Manager) are cleaned up immediately by Windows NT, rarely resulting in an orphaned session.

One common cause of orphaned sessions arises when a client computer loses power unexpectedly or is powered off without performing a proper shutdown. Orphaned sessions can also occur due to a "hung" application that never completely terminates, resulting in a dead connection. Windows does not know that the connection is dead and continues to report the action as active to SQL Server. SQL Server, in turn, keeps the session open and continues to wait for a command from the client.

Issues with Orphaned Sessions

Open sessions take up one of the SQL Server network connections. The maximum number of connections is limited by the number of server CALs; therefore, orphaned sessions may prevent other clients from connecting.

A more important issue is that open sessions use server resources and may have open cursors, temporary tables, or locks. These locks may block other connections from performing useful work and can sometimes be the result of a major pileup of locks. In severe cases, it can appear that SQL Server has stopped working.

The function sysprocesses (or stored procedures, such as sp_who) reports information on existing server sessions. Possible orphaned sessions can be identified if the status of a process is "awaiting command" and the interval of time found by subtracting last_batch from GETDATE() is longer than usual for the process. If the session hostname is known to be down, it is orphaned.

Windows checks inactive sessions periodically to ensure that they are active. If a session does not respond, it is closed and SQL Server is notified. The frequency of the checking depends on the network protocol and registry settings. However, by default, Windows NT performs a check only every one or two hours, depending on the protocol used. These configuration settings can be changed in the registry.

To close an orphaned SQL Server session, use the KILL command. All resources held by the session are then released. If orphaned sessions become a problem, registry settings can be changed on Windows to increase the frequency with which clients are checked to verify that they are active. Changing these settings affects other application connections and the following points should be considered before making any changes.

Resolving Issues through the Query Analyzer

At times, an application can create an orphaned session that holds locks on some tables and hence prevents other users from querying or writing to these tables.

A spid value of –2 is an indicator of connectionless or orphaned transactions. You can identify this in the sp_lock(spid column), sp_who or sp_who2 (blk column), syslockinfo and the sysprocesses table. You can use these SQL to find the orphaned sessions and resolve issues.

/*for finding locks and the spid*/
use master
go
exec sp_lock
go
/*for finding the process locking the records*/
use master
go
select * from sysprocesses where spid = ?
/*for finding the last statement being executed by the spid*/
use master
go
dbcc inputbuffer (spid)
/*for finding the name of the database in which the locks are being held
by the orphaned process*/
use master
go
select db_name(dbid) -- dbid is a column in the sysprocesses table
/*for finding the names of the objects on which the locks are being held*/
use
go
select object_name(object_id)

You may notice the spid value of –2. In this case you will have to use Kill UOW to kill the process. UOW is the unit of work of the DTC transaction and can be obtained from the syslockinfo table. You also may notice that the last statement that was being executed by the process is sp_cursorunprepare or sp_unprepare, which are API cursors. The situation in this case could be that the application forgot to clean up the session in the case of an error condition, hence creating an orphaned session on the server. In this case, you will have to use Kill spid to terminate the process (spid is the system process id for that particular process) and then fix the application code to close out the session in case of an error. You can also use the KILL spid/UOW with statusonly to check the status of the kill statement.

Changing the Check Frequency

Windows NT 2000 periodically checks inactive sessions to ensure they are active. If a session does not respond, it is closed and SQL Server is notified. The frequency of the checking depends on the network protocol and registry settings. However, by default, Windows NT performs a check only every one or two hours, depending on the protocol used. These configuration settings can be changed in the registry.

To close an orphaned SQL Server session, use the KILL statement. All resources held by the session are then released.

If orphaned sessions become a problem, registry settings can be changed on Windows NT to increase the frequency with which clients are checked to verify that they are active. Consider, however, the effect changing these settings may have on other applications on your system, in particular, applications with Internet connectivity, such as Microsoft Internet Information Services (IIS) or Microsoft Internet Explorer. In addition, consider the effects of using connections that are charged on a per-packet basis.

Warning

Do not change these settings on computers running Microsoft Windows 95 or Microsoft Windows 98.

Caution

Editing the registry is not recommended; inappropriate or incorrect changes can cause serious system configuration problems. Only experienced users should use the Registry Editor.

The registry entries can be altered from HKEY_LOCAL_MACHINE by double-clicking SYSTEM, expanding CurrentControlSet, and then clicking Services.

KeepAliveTime for TCP/IP

Key: Tcpip\Parameters

Value Type: REG_DWORD—Time in milliseconds

Valid Range: 1—0xFFFFFFFF

Default: 7,200,000 (2 hours)

Description: The parameter controls how often TCP attempts to verify that an idle connection is still intact by sending a keep-alive packet. If the remote system is still reachable and functioning, it will acknowledge the keep-alive transmission. Keep-alive packets are not sent by default; this feature may be enabled on a connection by an application.

SessionKeepAlive for Named Pipes

Key: Netbt\Parameters

Value Type: REG_DWORD—Time in milliseconds

Valid Range: 60,000—0xFFFFFFFF

Default: 3,600,000 (1 hour)

Description: This value determines the time interval between keep-alive transmissions on a session. Setting the value to 0xFFFFFFF disables keep-alives.

Do not increase the ping frequency to less than 1 minute, as network I/O and CPU usage for pings may become excessive. I increased the ping frequency to 2 minutes (down from 1 hour) and then pulled the plug on a machine that still had an open connection to the server. After 2 minutes the session was killed. Then I created 10 connections to the server and watched the network I/O and the CPU cost for the next 30 minutes using perfmon. The CPU usage peaked after every 2 minutes when Windows made a ping check for the servers. So it might not be such a good idea for the production servers, but if there is no other way for you to control the orphaned sessions, you can use this registry change to resolve the issues.


Linked Servers

SQL Server allows you to create links to OLE DB data sources called linked servers. After linking to an OLE DB data source, you can reference rowsets from the OLE DB data sources as tables in Transact-SQL statements and pass commands to the OLE DB data sources and include the resulting rowsets as tables in Transact-SQL statements.

Each distributed query can reference multiple linked servers and can perform either update or read operations against each individual linked server. A single distributed query can perform read operations against some linked servers and update operations against other linked servers. In general, Microsoft SQL Server requires distributed transactions support from the corresponding OLE DB provider whenever data from more than one linked server are likely to be updated in a transaction.

Though these things can also be done through the enterprise manager, I am sure that like me, you also like to use T-SQL to get these tasks achieved.

Creating Linked Servers

You need to use the following system stored procedures to create and configure the linked server definitions:

  • Sp_addlinkedserver

  • Sp_addlinkedsrvlogin

  • Sp_serveroption

  • Sp_dropserver

Syntax
Exec sp_addlinkedserver
@server = 'alias name of the linked server',
@srvproduct = '',
/*this is the product name of the OLE DB data source to add as a linked
server.*/
@provider = 'provider_name',
@datasrc = 'data_source',
@location = 'location',
@provstr = 'provider_string',
@catalog = 'catalog'

If it's SQL Server, then the provider_name, data_source, location, and catalog don't need to be specified.

Examples

  1. Creating a linked server for SQL Server:

    USE MASTER
GO
Exec sp_addlinkedserver
@server='DevServer',
@srvproduct='',
@provider='MSDASQL',
@provstr='DRIVER={SQL Server};SERVER=TESTSERVER;UID=sa;PWD=testcase:'

  2. Creating a linked server for Oracle:

    USE MASTER
GO
EXEC sp_addlinkedserver
@server = 'LONDON Mktg',
@srvproduct = 'Oracle',
@provider = 'MSDAORA',
@datasrc = 'MyServer'

Setting the Linked Server Options

You can use the sp_serveroption stored procedure to set the options for the linked server.

Exec sp_serveroption
@server = 'linked server name'
@optname = 'option name'
@optvalue = 'option value'

The important option names are:

  • Data Access: Enables and disables a linked server for distributed queries

  • Rpc: Enables remote procedure calls from the given server

  • Rpc out: Enables rpc to the given server

  • Collation Compatible: If this is set to true, SQL Server assumes that all characters in the linked server are compatible with the local server with regard to the character set and collation sequence.

Examples
Exec sp_serveroption 'DevServer', 'data access', 'true'
Exec sp_serveroption 'DevServer', 'rpc', 'true'
Exec sp_serveroption 'DevServer', 'rpc out', 'true'
Exec sp_serveroption 'DevServer', 'collation compatible', 'true'

Creating Logins for the Linked Server

Once you have added the linked server to the server, you need to create the logins for connecting to the server. The stored procedure sp_addlinkedsrvlogin creates or updates a mapping between logins on the local server and remote logins on the linked server.

Use Master
GO
Exec sp_addlinkedsrvlogin
@rmtsrvname='rmtsrvername', —this is the name of the linked server
@useself='useself',
/*True means that sql server logins are used, false means that the
rmtuser and rmtpassword are used*/
@locallogin='locallogin', —login on the local server, default is NULL.
@rmtuser='user_name' –name of the remote user
@rmtpassword='remote user password' –remote user password
Examples

  1. Connecting to the linked server using local logins

    Exec sp_addlinkedsrvlogin 'DevServer', 'true'

  2. Connecting to the linked server using remote user ID and password

    Exec sp_addlinkedsrvlogin
@rmtsrvname='DevServer',
@rmtuser='test_user',
@rmtpassword='test_password'

Querying the Linked Server

You can use two functions to query the linked servers:

  1. OPENQUERY: Executes a specified query on the linked server. It can be referenced in the from clause of the select statement.
Example
Select * from openquery(Devserver, 'select * from user_master')

The results from the openquery function can also be joined to another table's records in the query

Example
Select * from user_master
Left outer join (select login_user_id from openquery(DevServer, 'Select *
from user_master')) as test
On test.login_user_id = user_master.login_user_id

It's essentially the same as using the records from another table.

  2. OPENROWSET: This function includes all the connection information necessary to connect to the linked server and to query it. It can also be referenced in the FROM clause just like the openquery function.
Examples

  1. Use TestServer

    GO
 SELECT test.*
 FROM OPENROWSET('SQLOLEDB','ma-atl60';'sa';'MyPassword',
 'SELECT * FROM testdatabase.dbo.user_master ORDER BY user_lname,
user_fname') AS test

  2. Use TestServer

        GO
    SELECT test.*
    FROM OPENROWSET('MSDASQL', 'DRIVER={SQL Server};SERVER=MA
ATL60;UID=sa;PWD=MyPassword',
testdatabase.dbo.user_master) AS test
ORDER BY test.user_lname, test.user_fname

You can also query the linked servers using a four-part naming scheme:

Select * from linked_server.database.owner.objectname

I have found linked servers to be very useful. I have used them to write scripts to copy over the Oracle database to SQL Server and also for conversion of schemas from sybase to SQL server. Just for information's sake, Oracle has something similar, called heterogeneous services. The service needs to be configured to connect to the heterogenous source and then you can query it as a local database. The only problem is that Oracle's native ODBC driver for generic connectivity is not that robust, so you have to use third-party drivers from merant/intersolve which, though very good in performance, are costly. In Chapter 27, there are some sample scripts for comparing Oracle with SQL server schemas, SQL Server with SQL Server schemas, and Oracle with Oracle schemas, which expect the linked server definitions to be already present. You can use the preceding examples to create them.


Summary

In this chapter, we covered some advanced topics for the DBAs: DBCCs, linked server definitions, information schema views, and the controversial topic of orphaned sessions. DBCCs are a must for serious DBA/Developers to know for maintaining the sanctity of their systems, and the knowledge of linked server definitions is very useful for working with heterogeneous systems.


Chapter 23. Debugging Database Performance Issues

In this chapter we focus on troubleshooting with the tools provided with SQL Server. SQL Server 2000 has a very rich set of tools and utilities to help debug performance and applications, and in this chapter we focus on how to use the tools as well as how to get the information from the registry and the error logs to find problems.

I generally like to divide the debugging of performance into three broad categories:

  • server configuration (hardware)

  • SQL Server and database settings

  • Inefficient SQL/inefficient calls made to the database to retrieve data (Example: the cacheSizeProperty of the RecordSet Object in ADO)

The best strategies are using Performance Monitor and/or Task Manager to get an idea of where the bottleneck lies; using Profiler to find and tune inefficient SQL; and optimizing server and database settings.


Performance Monitor or Task Manager?

Which tool is best for you? Most likely, you'll use both the Performance Monitor and the Task Manager, depending on your mission. The Performance Monitor is the tool of choice for obtaining detailed information, logging data for extended analysis, and collecting performance information based on performance events that occur within your system. The Task Manager provides a quick look into what is occurring on your system but doesn't provide a mechanism for logging. However, the Task Manager lets you manage applications (that is, processes) that might be adversely affecting your system.


Performance Monitor

The Performance Monitor is a Microsoft management console (MMC) snap-in. On Windows NT, it runs as perfmon.exe, and under Windows 2000, since it's a snap-in of the Microsoft management console, it shows up as mmc.exe under Task Manager. To invoke the Performance Monitor, select Start, Programs, Administrative Tools, Performance. Alternatively, you can select Start and Run, input perfmon in the Open text box, then press Enter.

Windows 2000's Performance Monitor provides the following features to monitor and analyze your server's performance:

  • Realtime performance monitoring in chart, reporting, or histogram mode lets you highlight a counter on the Performance Monitor screen and press Ctrl+H, which highlights the current counter selection on your screen. After you perform this action, as you scroll through the counters, the Performance Monitor highlights in the associated graph each counter as you select it. When you're displaying multiple counters on the GUI, this feature helps denote which counter is doing what. (The Backspace key doesn't provide this functionality, as it did in NT 4.0.)

  • Trace logs provide an advanced mechanism to analyze your system. Third-party tools usually leverage this feature.

  • Counter logs let you log performance data at a designated interval for local or remote Windows 2000 systems.

In addition to these monitoring tools, the Performance Monitor provides enhanced functionality: Alerts let you generate an action (that is, run a command or script) based on the counter value thresholds you set in Performance Monitor. In addition, all your settings move with you from one reporting mode to another reporting mode. When you start the Performance Monitor, the tool recalls your last settings. Thus, you don't have to save your default settings to a .pwm file and recall them to begin analyzing your system. These setting are system based, so the next person who logs in will see the view you left. The new tool offers more flexibility in how you store the data the Performance Monitor generates (for example, you can store data as HTML, binary, .csv, .tsv, and binary circular) than do previous versions. You can start and stop performance logging based on a date/time group. You can automatically start another copy of the tools based on Performance Monitor events that you configure on your system. Finally, the new tool has a friendlier mechanism to simultaneously collect performance data from multiple servers.

Although NT 4.0 provides some of this functionality (if you install tools from the Microsoft Windows NT Server 4.0 Resource Kit), Windows 2000 provides these features in an integrated, friendlier tool that saves you the extra step of loading additional resource kit tools. In addition, Windows 2000's Performance Monitor can't read performance logs that you generate with NT's Performance Monitor.

Using the Performance Monitor

The following scenarios show you how to leverage the Performance Monitor's capabilities. To take full advantage of Performance Monitor's functionality, you must activate all your system's performance counters.

General performance monitoring

When you start the Performance Monitor, the tool presents you with the default Performance window. To add any combination of counters to the right display pane, click the addition sign (+) button in the toolbar at the top of the right display pane shown in Figure 23.1. When you're examining specific resources, include the appropriate counters for analyzing that area.

Figure 23.1. Addition of counters

In the Performance window, you can quickly change between chart, report, or histogram views by selecting the appropriate icon below the Performance Monitor's menu bar. Screen 1 shows an example of the report view. You can view the performance of a remote server by clicking the + button, selecting the "Select counters from computer" option, and entering the remote computer's name using the Uniform Naming Convention (UNC) format. (The Performance Monitor enters the name of the local computer by default.) You must have administrative rights on the remote system you want to monitor.

Long-term performance analysis

What if you want to collect performance information over time to develop a baseline? With NT 4.0, the biggest hurdle is the physical size the performance logs might grow to. To work around this limitation, Windows 2000 Performance Monitor lets you schedule log collection by time or date. This enhancement lets you isolate the collection of data to times of interest, thus lowering the amount of data the Performance Monitor collects. To set a schedule, expand the Performance Logs and Alerts object in the left pane of the Performance window, right-click a log, and select the Schedule tab. On the Schedule tab, you can configure start and stop times. Collecting performance data during typical operations (that is, from 8:00 A.M. to 6:00 P.M.) is common.

Depending on your environment, you might want to collect data for several weeks at a time for trend analysis. To avoid having to perform maintenance on these files as they grow, select Counter Logs, right-click the file you want to manage, select Properties, click the Log Files tab, select Binary Circular File from the Log File Type drop-down list, and input a limit in the "Limit of" text box, as Figure 23.2 shows.

Figure 23.2. Choosing limits and type of log files

Leveraging this performance-collection strategy lets you limit the amount of disk space a performance file uses. If you match the sampling rate to the amount of disk you want to use for performance collection, you can monitor and access several weeks' worth of performance data without worrying about performance log size maintenance.

Sampling rates

How often do you need to sample your system for performance data? The answer depends on your goals. If you sample more often than every 5 seconds, you place a slightly higher load (1 to 3 percent) on your system, and your performance log files require more disk space than if you sample at a usual rate (less often than every 5 seconds). If you don't sample often enough, you risk not monitoring the system when it experiences a problem.

Windows 2000 provides a much broader range of objects and counters than previously available. If you collect all possible performance data on a system with one disk and one network connection, each sample you collect requires more than 200 KB per sample. Most administrators don't need to monitor every possible performance object and its associated counters.

Which Process Is the Bottleneck?

You can monitor some of the performance counters (which we will discuss shortly), and they will provide you with your system's general performance baseline, but you'll need more data to determine which application or process is swamping your system. To obtain this information, use the Performance Monitor's alert feature to start any action based on an event you define (for example, when your counters reach their maximum performance thresholds).

For this example, set an alert to start a copy of the Performance Monitor counter logs when CPU usage exceeds 98 percent. (Occasional peaks in CPU usage might trigger this alert even when a problem doesn't exist. You can use third-party tools to start additional performance collection based on more advanced logical sequences—for example, when CPU usage exceeds 90 percent for 5 minutes, start additional performance data collection.) To configure this alert, start the Performance Monitor, expand Performance Logs and Alerts, and select Alerts. Right-click in the right pane, and select New, Create New Alert Settings, and a name. Add the counters you want to monitor and their threshold for triggering an action; select the Action tab, the "Start performance log" option, a counter log to start, and the Schedule tab; and fill in the times you want to run the monitor. Use a counter log that collects data from the counters and instances under the Process object.

With this setup, the Performance Monitor will alert you when your system has a performance problem, and the software will provide you with quantifiable and empirical data that illustrates which process is causing the problem. (The Performance Monitor will provide this information in the detailed counter logs that the tool started only after your system reached a certain threshold.)

The Performance Monitor's alert feature is flexible. You can tell the alert function to start any script or application. You can have the system send you an e-mail message or start a batch file that pings (that is, ping.exe), then trace routes (that is, tracert.exe) along the network path to a distant system with which you want to interact. In this manner, you can measure the network response time to determine whether your network has problems.


Task Manager

The Performance Monitor helps you track problems over time, but what can you do about problem processes in real time? The Task Manager provides mechanisms to monitor and resolve performance problems in real time. For example, say you have a hunch that a particular exe is your system's CPU hog. To activate the Task Manager, press Ctrl+Alt+Del and click Task Manager. Alternatively, you can run taskmgr.exe from the command line or right-click at the bottom of the bar and select Task Manager. After you start this tool, you can view numerous columns of performance data on the Processes tab. The amount of data available on Windows 2000's Task Manager Processes tab is much greater than the amount on NT 4.0's Task Manager Processes tab—in particular, finer-grain I/O information is available on a per-process basis (for example, I/O reads, I/O writes). Within the Processes view, you can quickly determine what amount of CPU, memory, and disk resources each process is consuming. The Applications tab lets you see which processes or applications are not responding.

To find out whether a particular exe is your system's CPU hog, select the Processes image name column to place the process list in alphabetical order. This action simplifies finding the culprit exe. After you find the filename, highlight it by clicking it, then right-click it. Task Manager presents you with several system control options.

  • End process: If the process isn't crucial, you can select this option and stop the process. This action might let another critical process get CPU time and operate properly.

  • End process tree: Some processes have multiple threads that might act independently. You can select this option to bring your system under control.

  • Set priority: You can select this option if you don't want to kill the process, but you want it to have less CPU time. Lowering the priority level of a process lowers the process's access to CPU cycles.

  • Set affinity: This option is available only on multi-CPU systems. You can select this option to bind an application to a specific CPU or set of CPUs to manage CPU usage. For example, if you have four, and one application is hogging all four, you can restrict the application to only one CPU so that other applications can use the freed resources.

You can lower regsvc.exe's priority by selecting Set Priority, BelowNormal, as Figure 23.3 illustrates. Figure 23.4 shows how you can monitor the CPU and memory utilization using Task Manager.

Figure 23.3. Setting the priority

Figure 23.4. CPU memory and usage
    I l@ve RuBoard


Performance Monitor Counters

Performance monitor counters can be divided into five categories:

CPU performance counters

I/O performance counters

Memory performance counters

Network performance counters, and

SQL Server performance counters

CPU Performance Counters

  • Object: Processor; Counter: % Processor Time. Measures the utilization of each individual CPU.

  • Object: System; Counter: % Total Processor Time. Measures average of all CPUs on the server. If it exceeds 80% for continuous periods (over 10 minutes or so), then there is a CPU bottleneck. The solution would be to get a faster CPU or to add more CPUs.

  • Object: System; Counter: Processor Queue Length. If the length exceeds 2 per CPU for continuous periods (10 minutes or so), then there is a possibility of a CPU bottleneck. So if a machine has four CPUs, this counter should not exceed a total of 8 for the entire server.

  • Object: System; Counter: Context Switches/Sec. This basically measures how often NT Server switches between threads.

    Heavy context switching hurts performance and should be minimized. The value of this counter should be less than 8000. If it nears 8000, probably NT Windows Fibers should be used instead of threads. Fibers are subcomponents of threads and have fewer overheads when switched between multiple CPUs in a SMP environment.

I/O Performance Counters

  • Physical Disk Object: Average Disk Queue Length. To monitor each disk array in the server. If this figure exceeds a value of 2 for each disk drive in an array, then there is a potential I/O bottleneck for that array. We need to calculate this figure since perfmon does not know the number of physical drives in an array. Example: For a particular array, if the value is 10 and the number of disks are, say, 6, then the ratio is 10/12 = 0.83. Before using this counter, we need to run "diskperf—y" to turn on the disk counters for the first time. Solutions to overcome the consequent performance issues are adding drives to the array, getting faster drives, getting a faster controller, and using a different version of RAID.

  • Physical Disk Object: % Disk Read Time and % Write Time. To gather information on whether the fill factor for the indexes needs to be adjusted. Over a period of time, the write/reads will yield good information in this regard.

Memory Performance Counters

I am assuming that the database server is dedicated to the SQL Server and perhaps some server-related utilities.

  • Memory Object: Pages/Sec. This counter measures the number of pages per second that are paged out of memory to disk or paged into memory from disk. Assuming SQL Server is the only major application running on the server, then this figure should average around zero, except for occasional spikes, which are normal. Over a continuous period of time (10 minutes or so), if this value is not nearly zero, the server is having to page data. SQL Server does not use the NT server's page file, so ideally if no other applications are running on this server, there should not be any paging going on on the server. If you are observing paging, there can be two reasons: You are running other applications on the server, which is causing NT Server to page, or you have set the SQL Server Max Server Memory Configuration setting to some value other than "Dynamically configure SQL Server memory." Ideally, SQL Server should not be competing for RAM with other applications.

  • Memory Object: Available Bytes. This counter is helpful in verifying whether SQL Server has enough RAM available. The counter can be viewed from perfmon or from the Task Manager (Performance Tab). The value should be more than 5 MB. If not, then we need more physical RAM. On a server dedicated to SQL Server, SQL Server attempts to maintain from 4 to 10 MB of free physical memory. The remaining physical RAM is used by the operating system. When the amount of available bytes is less than 4 MB, SQL Server is probably also paging (which it shouldn't be). When this happens, we can increase the physical RAM or change SQL Server's memory configuration settings.

Network Performance Counters

Before we can use the network counters, the Network Monitor Agent service must be installed on the server.

  • Network Interface Object: Bytes Total/Sec. This counter measures the number of bytes that are being sent back and forth between the server and the network in both SQL Server and non-SQL Server network traffic. If SQL Server is the only application running on the database server, then the vast majority of the traffic should be from SQL Server. There is no "correct number" for this counter. The best way to detect whether there is any network bottleneck is to compare it with the maximum traffic supported by the network connection our server is using.

  • Network Segment Object: % Network Utilization. This counter provides us with what percentage of the bandwidth is being used by the network connection our server is using. This is not the amount of bandwidth being sent to and from our server, but the total bandwidth being used on the connection the network card is attached to. This connection could be of many different types, including a shared hub or a switched port. An ideal case would be to have a network connection to its own dedicated switch port for maximum performance.

We can also find out how much data is being sent back and forth from the server to the network by using the server object bytes received/sec and bytes transmitted/sec.

SQL Server Performance Counters

One of the major reasons for poor performance is page splitting, which occurs when a data page or an index page becomes full and then is split between the current page and a newly allocated page. Some page splitting is normal, but excessive page splitting can cause performance problems. The SQL Server Access Methods Object is Page Splits per Second. If this number is high, then the fill factor for the indexes needs to be adjusted.

  • SQL Server Buffer Manager Object: Buffer Cache Hit Ratio. This counter shows how often SQL Server goes to the buffer, not the hard disk, to get data. This value should always be more than 95 percent. If not, more RAM should be added to the server.

  • SQL Server General Statistics Object: User Connections. This counter shows the number of user connections, not the number of users, that are currently connected to SQL Server. Please remember that a single user can have multiple connections and also multiple users can share a single user connection. Don't make the assumption that this number represents the actual number of users.

  • SQL Server Locks Object: Average Wait Time (ms). This counter monitors/measures the average wait time of a variety of locks: database, extent, key, page, RID, and table. This counter needs to be done in conjunction with SQL Server Profiler to find out what SPIDs are causing the locks.

  • SQL Server Access Methods Object: Full Scans/Sec. This counter yields information on how many table scans the database server is performing. This counter is for the entire server, not for a particular database. In this counter a number of scans appear periodically. These are the table scans that SQL Server does on a regular basis for internal use. What we should look for are random table scans that represent our application. Then we need to use the Profiler to break into the SQL and see if we need a better query plan. Of course, sometimes a table scan is faster than using an index and the optimizer might have chosen the correct plan.

Each SQL Server connection is associated with a SPID (SQL Server process ID). Current connection information (including the SPID) can be seen using the Enterprise Manager's Current Activity Window. On rare occasions, a user connection can virtually tie up an entire server's CPUs if something is wrong with the connection. In other cases, a long-running query may cause a performance problem. Unfortunately, the Current Activity Window does not always show that this is happening with individual user connections, although we can see that there is something wrong with the server because our CPU cycles are being used up if we use the Performance Monitor to monitor the System Object % Total Processor Time counter. So if we know that a user connection is causing problems but we can't identify it using the Current Activity Window, here's a method for identifing which user connection is causing the problem.

The first step is to verify that the SQL Server process (sqlservr)—not some other process—is responsible for the excessive CPU use. The sqlservr process is made up of many different threads, each one generally (but not always) representing a specific user connection.

To do this verification, we use the Performance Monitor and go to the Process Object: % Processor Time counter using the Chart View. When we do this, there will be many instances of running processes on the server listed in the Instance list box. We need to select all the instances from the Instance list box except the _Total instance (which just sums the total of all the other instances). Once the instances are displayed in the Chart View, we need to change the format of the chart from graph to histogram. This is done from Options|Chart on the Performance Monitor's main menu. Now we should be able to easily see which process is causing the problem. (Note: If "Idle" is the highest-rated process, then not much is going on with the server.) If SQL Server is the problem, then the instance with the highest percentage of processor time should be sqlservr. If this is the case, then proceed with the rest of this tip; if not, we need to find out what other process on our server is causing the problem.

Assuming that the sqlservr is the guilty party, the next step is to identify which user connection (SPID) is causing the problem. To do so, we use the Performance Monitor and go to the Thread Object: % Processor Time counter using the Chart View. Many instances of running threads on our server will be listed in the Instance list box. We select all the instances that begin with "sqlservr" from the list box. Once the instances are displayed in the Chart View, we need to change the format of the chart from graph to histogram. This is done from Options|Chart on the Performance Monitor's main menu. Now we should be able to easily see which thread is eating up the CPU resources.

While we have now identified which thread (and its Performance Monitor Instance number) is causing the problem, we still haven't figured out which user connection is causing the problem. Although we have identified a specific Performance Monitor Instance number as being the culprit, there is no correlation between a Performance Monitor Instance number and a SPID in SQL Server. If there were, our job would be much easier. Unfortunately, it will take us two more steps to make this correlation. So here goes.

The second step is to correlate the Performance Monitor Instance number we just identified with something called a KPID (kernel process ID) or an ID Thread. KPID and ID Thread are exactly the same thing. A KPID or ID Thread is a systemwide identifier that uniquely identifies a thread in NT Server. To do so, we use the Performance Monitor and go to the Thread Object: ID Thread counter using the Report View. Many instances of running threads on our server will be listed in the Instance list box. We select all the instances that begin with "sqlservr" from the list box, similar to what we did in the previous step. The reason we are using the Report View is that we need to find out which Thread ID (KPID) matches the Performance Monitor Instance number (this is the number identified in the column headings in the report) we identified in the previous step. Now that we know this, we are only one step away from finding our culprit.

The last step is to correlate the Thread ID (KPID) identified in the second step with the SPID. To do so, we run the following query in Query Analyzer:

Use master
select spid, kpid, hostname, dbid, cmd from master..sysprocesses order by kpid

This query will display a report listing both SPIDs and KPIDs. And this is where we make the last correlation. Just match up the Thread ID (KPID) found in the last step, and then match it to the corresponding SPID, and now we know which SQL Server user connection is causing the problem.


Settings

On a multiple CPU machine that is running to nearly 100 percent utilization, and assuming that there are no other viable options to reduce CPU utilization, we may be able to improve performance by using NT Windows Fibers. Fibers are subcomponents of threads that perform similar to threads. The advantage to using them is that they have fewer overheads when being switched between CPUs in a multiple CPU server. The benefits show up when the server's CPU is running at near maximum or the perfmon system Object Context Switching/Sec counter exceeds 8000 switches per second for some time (10 minutes or so). So if the CPU is not maxed out, don't change this option. It is available under SQL Server Properties, Processor tab.

If there are a lot of insert, update, and delete statements, the default "recovery interval" of 0 (which means that SQL Server determines the appropriate recovery interval) may not be appropriate. If regular periods of 100 percent disk-write activity are occurring through the perfmon (through the checkpoint process), the "recovery interval" should probably be set to a higher number, such as 5.

The smaller tables that are mainly used for read access only should be cached into the memory. This will reduce the I/O.

If the server has plenty of physical RAM, many memory-intensive queries are run that make extensive use of hash and sort operations, and many queries are running concurrently on the server, we can consider boosting the "min memory per query" SQL Server configuration option from its default of 1024 K to a higher value. This setting tells SQL Server to automatically allocate, at a minimum, the amount of memory set in this configuration setting. Increasing this setting can boost the performance of memory-intensive queries. On the other hand, if we increase this value too much, queries could get slower, because SQL Server will not run the query until it can allocate the amount of memory specified in this setting or until the value specified in the query wait SQL Server configuration option is exceeded. We need to test this setting with several different values until we come up with the ideal setting for our particular SQL Server environment.

If the server is dedicated to SQL Server and is running more than one CPU, then we can consider setting the priority boost SQL Server configuration setting from the default of 0 to 1. This gives sqlservr.exe a higher Windows NT 4.0 scheduling priority than other processes running on the server. On a multiple CPU server, it changes the process priority from 15 to 24, giving SQL Server a little performance boost. Before using this setting in production, we should test the server's performance before and after the change to see if SQL Server benefited from the change.

If the server is dedicated to SQL Server, we need to take a look at the Max Worker Threads SQL Server configuration setting, checking to see if it is set appropriately. The default setting is 255, and it is used to determine how many worker threads are made available to the sqlservr.exe process from Windows NT 4.0. If the number of user connections is less than 255, then each user connection uses one worker thread. If there are more than 255 user connections, then SQL Server uses thread pooling, where more than one user connection shares a single worker thread. Although thread pooling reduces the quantity of system resources used by SQL Server, it can also increase contention between the user connections for access to SQL Server. For best performance, we want to set the Max Worker Threads setting to a value equal to the maximum number of user connections your server ever experiences plus 5. If the number of user connections (plus 5) is less than 255, resources are freed up for SQL Server to use elsewhere. If the number of user connections (plus 5) is greater than 255, we don't have to worry about user connections contending for resources. But keep in mind that the additional worker threads use physical RAM in the server. So if the physical RAM on the server is already maxed out, the Max Worker Threads should not be increased more than the default of 255. If the physical RAM is available, then it can be taken advantage of by increasing the Max Worker Threads as described.


Event Viewer

Windows NT and Windows 2000 also have a common interface for posting and reading application, system, and security events. The event viewer in the Administrative Tools program group displays recorded events for all three logs. You can also start Event Viewer from Start/Run, then typing "eventvwr" and pressing Enter. These events represent both failures and successes as well as simple information. You can set up SQL Server to log events to the Windows NT event log, which logs status during startup and provides runtime errors and warnings. Also, applications running with SQL Server can log application-specific error messages into the event log from within stored procedures and triggers; these messages give you information you can use to completely customize SQL Server database applications.

The SQL Executive uses the Windows event log to trap errors for alerts and notifications; this integrated notification gives you exceptional monitoring ability for both the local system and remote systems. You can use SQL Executive to create alerts and notifications based on errors logged to the Windows event log. Using this process, SQL Server and SQL Executive can significantly enhance Windows's notification process and provide remote notification for almost any system or application event.

You can also examine the event log on remote machines through the Event Viewer. SQL Executive can respond to events posted in those locations using locally stored alert configurations, thereby centralizing alert handling and notifications. These alerts need not relate only to administrative tasks; they can easily be extended to program-specific notifications, such as stocking requirements or large transactions on a sales system.

Figure 23.5 shows the three types of logs that you can monitor: Application Logs, Security Logs, and the System Logs. You can look at the detailed information and the time the error/warning/information was logged, as shown in Figure 23.6. If you do not want to see the warnings or the informational messages, you can turn those counters off, as shown in Figure 23.7 (you can select this option by right clicking the log and selecting Properties).

Figure 23.5. Event Viewer options

Figure 23.6. Event properties

Figure 23.7. Application Log properties
    I l@ve RuBoard


SQL Server Profiler

You can get a lot of information from the SQL Server Profiler, so you should be careful about which data you really want to pick out. Also, SQL Server 2000 ships with some trace files of its own that are very helpful in resolving issues. The counters can be monitored and appropriate action taken based on the query plans and other information yielded by the trace. One of the trace files I prepared is included as a part of the CD that will help in troubleshooting the issues with the application. A detailed description on the Profiler is in Chapter 6.


Hardware Issues

The following are general recommendations. Of course, the size of the database will also determine the configuration settings.

Ideally, the database server should be a stand-alone server to avoid sharing hardware resources. The same box should not be used as a Web/App server as well unless the databases are pretty small and there is not much traffic.

NTFS formatted partitions should not exceed 80 percent of their capacity. For example, if we have a 20-GB drive, it should never be fuller than 16 GB because NTFS needs room to work, and when we exceed 80 percent capacity, NTFS becomes less efficient and I/O can suffer for it. We can set up an alert in the SQL Server agent to notify us if the arrays exceed 80 precent of their capacity.

SQL Server can take great advantage of multiple CPUs. For example, SQL Server can take a single query (if it meets certain criteria) and split it between multiple CPUs, greatly speeding performance. So we should aim for a server with room for at least two CPUs.

We should select a CPU with a large L2 cache, especially important if we are using multiple CPUs. The rule of thumb is 1-MB L2 cache for one or two CPUs and at least a 2-MB L2 cache in each CPU for four or more CPUs. The greater the L2 cache, the greater the server's CPU performance because it reduces the amount of wait time experienced by the CPU when reading and writing data to the main memory.

Like server hardware, user's hardware can affect the performance of SQL Server-related activity. At a minimum, the workstations should have a 200-MHz CPU and a 128-MB RAM.

The more physical RAM a server has, the better the SQL Server performance. Windows 2000 Server and SQL Server 2000 Standard Edition can use up to 2 GB of physical RAM. Windows 2000 Server and SQL Server Enterprise Edition can use up to 4 GB of physical RAM, Windows 2000 Advanced Server and SQL 2000 Enterprise Edition can use up to 8 GB of RAM, and Windows 2000 Advanced Server with SQL 2000 Enterprise Edition supports 64 GB of RAM.

Hardware-based RAID is preferable to software-based RAID because the latter is slower; it can't offload the work to a separate processor, as does hardware-based RAID solutions. Use RAID 1 for the tempdb and Database Logs for faster I/O and RAID 1+0 for the database files for mirroring and striping. For a much greater load, use a RAID array made up of two or more physical drives that offer fault tolerance, instead of storing the operating system, application software, or databases on single disk drives.

A top-notch I/O controllers offloads much of the I/O work onto its own local CPU, freeing up CPU time on the server to do other tasks. A fiber channel connection instead of an SCSI connection is ideal.

The server should have at least one 100-MB network card and perhaps two. Two cards can be used to increase network throughput and to offer redundancy. In addition, the network card(s) should be connected to full-duplex switched ports for best performance. The network card(s) on the server should be set to the same duplex level as the switched port they are connected to (assuming they are connected to a switch and not a hub).

The network libraries we choose when installing SQL Server can affect the speed of communications between the server and its clients. Of the three key network libraries, TCP/IP is the fastest and Multiprotocol is the slowest. Because of the speed advantage, TCP/IP should be used on both servers and clients. Also, unused network libraries on the server contribute unnecessary overhead.

Don't install SQL Server services you don't need, such as Microsoft Search, OLAP, or English Query, as they only add overhead to your server. If the application doesn't use the Microsoft Distributed Transaction Coordinator (MS DTC), turn this service off by setting it to manual using the Services icon in the Control Panel.

To summarize, here are suggestions for resolving SQL server hardware bottlenecks.

I/O Bottlenecks

  1. Add physical RAM so that your server can go to RAM instead of the I/O system to access data.

  2. Use RAID level 5 or RAID level 10 for your arrays if you do not do so already. RAID level 10 is the fastest RAID level you can choose that supports redundancy.

  3. Add physical drives to the current arrays. This helps to boost both read and write access times. But don't add more drives to the array than your I/O controller can support.

  4. Replace your current hard drives with faster drives.

  5. Add faster or additional I/O controllers. Consider adding more cache (if possible) to your current controllers.

  6. Tune your application so that it doesn't have to access the disk as often. For example, add indexes so that table scans aren't needed, normalize your database to eliminate redundant data, and so on.

  7. Move database or transaction log files from busy arrays to less busy arrays.

  8. Store databases and transaction log files on a SAN (storage area network).

  9. Use partitioned views and federated servers to distribute workload.

  10. Ensure that SQL Server is the only application (other than server management utilities) running on the server.

  11. Remove or disable all unnecessary services.

CPU Bottlenecks

  1. Get faster CPUs, or add CPUs.

  2. Get CPUs with a larger L2 cache.

  3. Tune your application so that SQL Server does not have to access the disk as often. For example, add indexes so table scans aren't needed, normalize your database to eliminate redundant data, and so on.

  4. Tune your queries to reduce the CPU load.

  5. Move some of the processing load to another SQL Server.

  6. Consider turning on Windows NT fibers.

  7. Be sure that both OLTP and OLAP queries are not being run on the same server. These different database applications should be performed on separate servers.

  8. Ensure that SQL Server is the only application running on the server (other than server management utilities).

  9. Remove or disable all unnecessary services.

Memory Bottlenecks

  1. Add physical RAM. If you are already at 2 GB of physical RAM and are still having a memory problem, consider upgrading to SQL Server 7 Enterprise Edition, which supports up to 3 GB, or to SQL 2000 Enterprise Edition, which in theory can support up to 64 GB of physical RAM.

  2. Ensure that SQL Server is the only application running on the server (other than server management utilities).

  3. Remove or disable all unnecessary services.

  4. Ensure that SQL Server is running as a member server (not a domain controller).

  5. Configure SQL Server to allocate memory dynamically (the default); don't hard code the amount of RAM allocated to the SQL Server.

Network Bottlenecks

  1. Add faster network cards.

  2. Add additional network cards.

  3. Server network card should be attached to switches.

  4. Network cards should be running in full duplex mode.

  5. Tune your application so that it does not require unnecessary network trips. Do this by returning only the required data and used stored procedures.

  6. Remove all unnecessary network protocols from the server.

  7. Ensure that SQL Server is the only application running on the server (other than server management utilities).

  8. Remove or disable all unnecessary services.

  9. Use TCP/IP as the network library on the client and server.


Database Setup

Locate the database files (.mdf) and log files (.ldf) on separate arrays in your server to isolate potentially conflicting reads and writes. To store database files (.mdf), use RAID 10 arrays for best performance. If this is too expensive, then RAID 5 is the next best bet. As many physical disks should be in the array (whether 5 or 10) as the controller will support. This allows reads and writes to be performed simultaneously on each physical drive in the array, significantly boosting disk I/O.

To store database log files (.ldf), use a RAID 1 (mirrored or duplexed) array. If there is only a single log file on the RAID 1 array, the file can be written to sequentially, speeding up log writes. But if there are multiple log files (from multiple databases) sharing the same RAID 1 array, then there is little or no advantage to using a RAID 1 array. This is because although writing to a log is done sequentially, multiple log files on the same array means that the array can no longer write sequentially but has to write randomly, negating much of the benefit of a RAID 1 array. But note: A RAID 1 array might still be better than a RAID 5 array if it can process more random writes than the RAID 5 array. Check your hardware to know for sure. Another option is to put each database log on its own separate RAID 1 array. One more option is to put the log on a RAID 10 array, which offers the best features of RAID 1 and RAID 5. Although this practice is expensive, it does provide optimum performance.

If your database is very large and very busy, multiple files can be used to increase performance. Here is an example of how you might use multiple files. Let's say you have a single table with 10 million rows that is heavily queried. If the table is in a single file, such as a single database file, then SQL Server would use only one thread to perform a sequential read of the rows in the table. But if the table is divided into three physical files (all part of the same filegroup), then SQL Server would use three threads (one per physical file) to sequentially read the table, which potentially is much faster. In addition, if each file were on its own separate disk or disk array, the performance would be even better. Essentially, the more separate physical files that a large table is divided into, the greater the potential performance. Of course there is a point where the additional threads aren't of much use when you max out the server's I/O, but up until you do max out the I/O, additional threads (and files) should increase performance.


SQL Tips

In my experience, 80 percent of performance gains on SQL Server come from making improvements in Transact-SQL code, not from devising crafty configuration adjustments or tweaking the operating system. The first optimization task is to decide which component of a system takes priority in optimization efforts. Your goal may be to achieve faster response time, add more concurrent users, improve the overall number of transactions per day, or achieve a combination of objectives. Remember: Improvements in one area often require compromises in others.

SQL Server has a built-in query optimizer that makes important decisions about query execution. It chooses the order in which tables are joined, which index to use on each table, and whether to employ special tricks such as creating worktables and breaking queries into steps. In most cases, it optimizes queries just fine. The Transact-SQL command SET SHOWPLAN ON allows you to view the query plan chosen by the optimizer. If you don't like it and want to enforce your own plan, then you can use special Transact-SQL commands and so-called optimizer hints to override the optimizer's decisions. Other commands allow you to display performance statistics, analyze index efficiency, and evaluate the cost of all query execution plans.

Create and Use Proper Indexes

A good index may improve query performance by a factor of 1000. Creating proper indexes is the most critical aspect of Transact-SQL performance optimization. Very often, a new index can reduce the response time of a query from several hours to several seconds. When this happens, you have your moment of programmer's glory. Everybody notices the dramatic improvement, and it takes no extra hardware resources. Managers and users love it! But there is another side to this coin. Poorly chosen indexes result in horrible response time and immense aggravation of the same managers and users.

The SQL Server optimizer can analyze and choose indexes for you, but it can choose only from existing ones. It is difficult to overstate how important good indexes are to query performance. Spend extra time analyzing and creating the best indexes for any query that you expect to show good performance.

Before we compare different index types and options, we need to realize that creating more indexes is not always better. Every new index on a table may improve some of the SELECT, INSERT, UPDATE, and DELETE queries but may slow down every other INSERT, UPDATE, and DELETE transaction on the same table. This is because SQL Server has to update every index to include or delete keys of affected rows or change pointers to pages where updated rows reside if they change location. The slowdown is usually within one or several percentage points of the transaction response time, but these taxes add up if you create many indexes.

In some companies, creation of indexes is a DBA's prerogative. But programmers are the ones who write SQL code that uses them. Very often, they have a better idea of what index they need for a particular transaction, while a DBA has a general view of the database and SQL Server as a whole.

Decision-support applications and data warehouses typically require more indexes per table to satisfy all possible queries. They also are tolerant of slower modification queries, because those are limited to data loads and usually run during nonpeak hours as batch jobs. At the other extreme are OLTP applications that are very sensitive to decreases in performance of INSERT, UPDATE, and DELETE transactions. Indexes require special maintenance in databases with volatile data. If you insert, delete, and update all or a large percentage of records in a table, the DBA must schedule an UPDATE STATISTICS job to keep indexes useful. You may even experience dramatic drops in query response times. Suddenly the stored procedure that took a second yesterday takes 10 minutes today. Users are unable to perform their work. If you see these symptoms, find out which table is constantly locked (run system stored procedure sp_lock) and then ask the DBA to update the statistics on that table. Very often, doing so "miraculously" puts your performance back on track. In general, new indexes improve the overall performance of an application. For example, consider an application in which the addition of a new index could significantly improve a rarely executed report query at the expense of marginally slowing down frequently executed update transactions.

Use Clustered Indexes

The SQL Server optimizer dearly loves clustered indexes. They are more efficient for many types of queries than nonclustered ones. The trouble is that you can't have more than one clustered index per table. For example, if you have two important queries on the same table using different keys in the ORDER BY clause, you have to choose which one is more critical. You can create a clustered index to optimize one report and a nonclustered index for another. It's possible that the nonclustered index will not be very efficient for the second report and won't even be chosen by the optimizer.

In such conflict situations, you have to consider carefully which query is more important to optimize first. It is important to have a clustered index on every table. Clustered indexes help manage database space more efficiently. Tables without a clustered index have a tendency to expand over time and use more space for the same number of rows. Current versions of SQL Server have automatic garbage collection on the page and extent level. A database page allocated to a table may be reused by the same table when all rows from it are deleted. An extent may be reused by any object once all eight pages of it become empty. But as long as a single row remains on a page of a table without a clustered index, it cannot be reused to insert new data. The only exception is the very last page of the table. All new rows are added to the bottom of the table. Compare this situation to tables with a clustered index, where new rows may be inserted in the middle of a table and added to partially used pages.

Avoid creating a clustered index on a monotonically increasing key. It forces all the new rows to go to the end of the table. Examples of such a key are columns with the IDENTITY property, columns used to store the current date and time, and TIMESTAMP columns. Such a clustered index forces all users inserting data to compete for the last page of the table. This may cause last-page locking contention—also known as a hotspot. Most disappointing is the fact that such indexes are usually less efficient than nonclustered ones. If you have an IDENTITY column, you are not likely to search for all rows given a range of keys but will probably often select a specific row by its IDENTITY key. It is also rare that you would sort by this column since it has no business meaning. A nonclustered index on such a column is typically better, because it's almost as efficient as a clustered one in terms of access and allows you to create a clustered index for some other purpose.

Clustered indexes are good for queries that use

  • GROUP BY clauses that use all or the first few columns of the clustered index key.

  • ORDER BY clauses that use all or the first few columns of the clustered index key.

  • WHERE clause conditions comparing the clause to the first or the first few columns of the clustered index key and retrieving many rows.

Long keys (either based on long columns or composite keys comprised of many columns) are not good. A clustered index on a long key takes no extra space for the leaf level. A nonclustered index on a long key may be quite large, because a lot of space is needed to store keys on the leaf level of the index. The first three types of the index benefit from the fact that requested rows are located together on consecutive rows of the table. SQL Server has to find only the first qualifying row and then keep on scanning until all rows are done. Several rows found on a single page reduce the number of I/O operations needed to access all the data. Additionally, Microsoft SQL Server has a performance booster—the read-ahead feature that automatically detects sequential reads from the same table and prefetches data pages before the query asks for them.

With a nonclustered index, it usually turns out that requested rows are scattered around different pages, and it may take a separate I/O operation to get every row. Therefore, a query using a clustered index is much more likely to take advantage of read-ahead. Nonclustered indexes are good for accessing one or a few rows. When you retrieve one row, you cannot take advantage of multiple rows located on the same page. They also are efficient for queries that can find all columns necessary to produce results in an index. In such cases, SQL Server doesn't even look at data pages. It uses indexes to process the whole transaction. Most indexes have keys narrower than the underlying table and take less space than to scan the tables to which they belong. Therefore, it takes fewer I/O operations to scan the index than to scan the table, and the whole transaction has a better response time.

Clustered indexes usually have one less B-tree level on top of the leaf data pages than do nonclustered ones. In such cases, it takes one extra I/O to access a row via a nonclustered index compared to a clustered one. But the benefit may be negligible when index pages are found in cache, and this extra I/O is an inexpensive logical one (in data cache).

Short Index Keys

Indexes with short keys are generally more efficient than those with long keys, because more keys fit on a single page and the whole index takes less space. Consequently, the index is faster to scan and more likely to fully fit into the data cache. Of course, columns that make up an index key should be chosen based on application requirements and the underlying data model. Very often, you have a choice of adding an extra column to the key that doesn't make a big difference in index selectivity. It may be more efficient to make the index leaner and meaner rather than use long composite keys, even though they may be more selective. A longer key means that fewer keys fit on a single database page, and the index may require more levels in the B-tree.

As a result, it takes more I/O operations to get the same data through this index, and transactions slow down.

Use Covering Indexes

Sometimes it pays to create an index with a long composite key that includes all columns needed to process a certain critical query. A nonclustered index that includes (or covers) all columns used in a query is called a covering index. When SQL Server can use a nonclustered index to resolve the query, it will prefer to scan the index rather than the table, which typically takes fewer data pages. Provided that an index key is not too long, SQL Server can place more keys than actual table rows on a single page. The whole index takes fewer pages than table data. If your query uses only columns included in the index, then SQL Server may scan this index to produce the desired output. For example, suppose that your application allows a wildcard search on the employee last name:

SELECT lname FROM employee WHERE lname LIKE '%ttlieb%'

This query does a table scan on the employee table. But if we create the following covering index, the query becomes more efficient:

CREATE INDEX employee_lname ON employee ( lname )

In this case, the table is small and the data occupies only two pages. But the index employee_lname is even more compact; it takes only one page. Therefore, with the help of the covering index, the same query requires only one physical I/O operation instead of two. In this case, if the query uses several columns, the covering index has to include them all, as in the following example:

SELECT          fname, lname
FROM            employee
WHERE           lname LIKE '%smith%'
AND             job_lvl < 200

A covering index for this query has to include three columns:

CREATE INDEX employee_name_lvl ON employee ( lname, fname, job_lvl )

It is important not to overdo your indexes. If a covering index benefits only a single query but takes a toll on every INSERT, UPDATE, or DELETE transaction on the same table, then it may not be a good idea to create the covering index. Making covering index keys too wide reduces their benefits, because an index on a wide key takes more space and may not be much more effective than scanning the data pages of the table.

Help SQL Server Choose Indexes

The mere existence of indexes is not enough for SQL Server to use them. You have to specify conditions in the WHERE clause that take advantage of indexes. If you have only partial conditions for a composite key, specify WHERE clause conditions for the leftmost columns of the key. For example, suppose you have a table with an index on columns a, b, c, and d. Depending on your WHERE clause conditions, SQL Server may use all or fewer columns of the index or not use the index at all (Table 23.1).

An index cannot be used unless all columns needed for the query can be found in the index, in which case SQL Server may do an index scan instead of a table scan.

Estimate Index Selectivity

Index selectivity measures the duplication of index key values in the table. You can say that an index with unique keys has perfect selectivity. The opposite is an index in which all key values are the same. It has the worst possible selectivity and is obviously useless. The selectivity of an index is a measure of how many different index keys are found in the table related to the total number of table rows. The following formula estimates the selectivity of a particular index:

Index selectivity = Number of distinct keys in the index / number of table rows

The closer this mark is to 1, the better for the optimizer.

The existence of duplicate keys lowers selectivity and reduces the effectiveness of the index. As a rule of thumb, nonclustered indexes with selectivity less than 0.1 are not efficient, and the optimizer refuses to use them for queries. It may take more I/O operations to complete a transaction using a nonselective index than to do a table scan. Exceptions may be made in special cases in which all columns needed for the query are found in the index and access to data pages is not needed.

Table 23.1. Usage of composite key columns
WHERE clause conditions key | Columns that may be used
WHERE a = @a AND b = @b AND c = @c AND d = @d | a, b, c, d
WHERE a = @a AND b = @b AND c = @c | a, b, c
WHERE a = @a AND b = @b AND d = @d | a, b
WHERE a = @a AND c = @c AND d = @d | a
WHERE b = @b AND c = @c AND d = @d

Microsoft SQL Server also measures index density, which is inversely related to selectivity. In other words, the higher the selectivity of an index, the lower its density. Use the DBCC SHOW_STATISTICS command to obtain index density information.

The density of an index with a composite key may be measured for all columns of the key, as well as for the first one, two, or more columns. The density of partial keys is important in cases in which only the first few columns of the key appear in a WHERE clause and SQL Server has to decide whether to use the index. Nonselective indexes aren't always bad. They may be efficient when they cover all columns of a query. For example, suppose that we add a column called gender of type CHAR(1) to the employee table and store values of M and F in it. You could then write a query to calculate the number of male and female employees:

SELECT  gender, COUNT(*) FROM employee GROUP BY gender

Without an index on the gender column, the optimizer has to scan the whole table to find the answer. If you create an index on this column, it may improve the query performance:

CREATE INDEX employee_gender ON employee ( gender )

Even though the index is very nonselective and is not good for any other task, your specific query is optimized by scanning the index, which takes much less space than the whole table. The benefits of following this approach should be carefully weighed against the increased transaction cost required to maintain the index, as well as the database space it takes.

Use the SQL Server Query Optimizer

The query optimizer is your invisible partner in Transact-SQL programming. It analyzes every query you submit to SQL Server and chooses the best execution plan for it. It is quite intelligent, very fast, never complains about night work, never asks for a raise. It may sometimes make a mistake, just as humans do, and may come up with an inefficient execution plan. There are ways to give it direct orders, and it follows them precisely, regardless of what it thinks of your decision. If it were human and had some sense of humor, it would be a perfect partner to work with. Partnership means that you have to do your share of work. The optimizer is not flawless. It may overlook the best solutions, and it is not smart enough to remember past mistakes. It cannot go beyond existing indexes, and it does not recognize obsolete index statistics.

The good news is that you can help the query optimizer. Every Transact-SQL query requires SQL Server to create and execute a little (or not so little) program. There may be numerous ways to execute the same query. The query optimizer has to choose the best join order, which indexes to use for search arguments and join conditions, and whether to create worktables. All these choices result in the creation of a query execution plan—an internal program that SQL Server executes to process your query. SQL Server processes every query in several steps. It parses the query, compiles the best (in its educated opinion) execution plan, and places the chosen plan in the procedure cache. Then it executes the query. Once the query is complete, SQL Server removes the plan from the cache.

Stored procedures, views, and triggers are handled differently. The first time somebody invokes an object after SQL Server is restarted, a plan is created for this object and placed in the procedure cache. The plan for a stored procedure depends on the actual values of parameters passed to the stored procedure at the first execution. The plan doesn't disappear from the cache after execution is complete. It may be used again and again by subsequent calls. Procedures created with the WITH RECOMPILE option are an exception, because their plans are recompiled and reoptimized at every execution.

When a user executes a stored procedure, SQL Server checks the procedure cache and, if there is a plan and it's not used by another user, that plan is reused. But if two users attempt to use the same object, SQL Server compiles another instance of the plan. This is because procedures are not reentrant and each user needs a separate copy of the plan. The second plan may be different from the first one, because parameters passed to the procedure may be different and table statistics may have changed since the first plan was compiled. If more than two users execute the same object concurrently, then multiple instances of the plan are created. All plans are dropped from the procedure cache when SQL Server shuts down.

The system table sysprocedures contains a parsed tree of each stored procedure, trigger, and view. It has references to databases and other objects resolved, but it's not compiled and optimized. The SQL Server procedure cache contains completely compiled and optimized procedure plans. At this point, all join orders are defined, indexes are chosen, and decisions are made on the use of worktables and execution methods.

Use Indexes Effectively

Indexes are created on all primary key and foreign key columns. SQL Server, by default creates indexes on primary key columns. We shall create indexes on foreign key columns, the columns most likely to be used in join conditions and limiting conditions.

Use Foreign Keys for Joins

When joining tables, favor using the relationship specified in the model to join. For example, to join child_table and parent_table tables, use child_table.parent_id and parent_table.parent_id as keys to join. Any other keys used in the join may not use the indexes, and the join may end up doing full table scans. Functionally also it may not fetch the desired record set.

Use Concatenated Keys Correctly

If multiple columns are part of a key, you must use the leading columns of the key in the WHERE condition in order to use the index. For example, the following query does not use primary key index (col1, col2, col3, col4) of the tableA table because col3 is not used in the WHERE condition.

Select ... from tableA
Where col3 = 'xyz'

Use Only Selective Indexes

The indexed column on a table should have a high selectivity ratio. Given a key column, if it is going to select more than 30 percent of the table records, it is not a good idea to use the index. For example, if a table has 1000 rows and an indexed column has 950 distinct values, the selectivity ratio is 950/1000, or 0.95. In this case it is better to use the index to select records. Note that the primary key index will always have a selectivity ratio of 1, so it is the best candidate for use in the WHERE condition.

The main purpose of using an index to retrieve records is to minimize the I/O on a table. For a column value, if an index is going to select more that 30 percent of the table records, doing a full table scan would be better than hitting the table through indexes. To use an index to retrieve records, the database has to do two disk reads, one to read index pages and one to read data pages.

For example, let us assume that there are 1000 records in Test table with Test_ID column indexed. Out of 1000 records, 100 records are in 10 status (unselected), 800 are in 90 status (invoiced), and other records are in other statuses. In this case, having a WHERE condition to select invoice status records is not efficient because it is going to select 80 percent of the total records. Here it would be better to do a full table scan. But to select status-10 records; it is better to use an index.

So, before deciding to use an index or create an index, you need to understand the record distribution pattern for key columns for various values. Note that even if a column has an index, you can force an SQL not to use an index by having a dummy operation on the column. Example:

Select...
From Test
Where Test_ID + 'A' =  'IA'

Avoid NULL Checks

The NULL checks in the WHERE condition will not use an index. If a key column is going to be frequently checked for NULL in the WHERE condition, have a default value for the column instead of storing NULL in the database. This is applicable only for large tables where a full table scan is expensive. The NULL checks are written as:

  • Where column is NULL, or

  • Where column is not NULL

Avoid Functions and Expressions on the Indexed Columns

If functions or arithmetic expressions are performed on a column in the WHERE condition, then the index on that column, if there is any, is not used.

Even if there is no index currently on a column, not using functions or calculations is recommended, since an index may be added to this column at production time to boost performance. For example, the following queries will not use index on the col3, if there is one:

select .... from Test
where col3 + 'X' = 'IX'
select ... from Test
where isnull(col3,'X') = 'A'

Instead of having functions/operations on the database columns, try to have it on the host-variables or constants. For example, the following query will not use index on col3:

select ... from Test
where col3 + 50  < :qty

But the following query will use the index:

select ... from Test
where col3 < :qty - 50

Avoid the Inequality Operator

Inequalities will not use the index—for example,

Where column !=  'x'

Instead try to have

Where column > 'x' or
                       column < 'x'

Avoid Full Table Scans

Full table scans sequentially read all data from a table, whether your application needs all data or not. Full table scans are not selective. Typically in an OLTP application, business operations are performed only on selective records, not on all the records of a large table. If an SQL needs to read all the records, the design should probably be revalidated. For example, a condition like TestID > 0 reads all the records from tableA. In an OLTP environment, reading all the records with > some condition may not be the actual requirement, though this kind of read may be required for batch operations.

Use EXISTS to Check for the Existence of a Record

Do not use select count( [image]) to check for existence of a record in the database. It is not particularly efficient, since it has to count all the records matching the WHERE condition before it returns the result to the program. Instead, use EXISTS to return the result immediately after it hits the first record matching the WHERE condition.

Replace NOT IN with NOTEXISTS

NOT IN clauses are expensive since they force the optimizer to do nested table scans. NOT IN queries can be rewritten with either NOT EXISTS (in which case the optimizer can use the indexes if present) or by doing a left outer join and checking for a NULL condition. For example,

Select col1, col2,.... from tableA where col1 not in (select col2 from tableB)

can be rewritten as

Select col1, col2,.... from tableA where not exists (select col2 from tableB where
tableB.col3 = tableA.col1)

The optimizer in this case can utilize the index present on col3 column and avoid the nested table scan. The query can also be written as

Select col1, col2, .... from tableA left outer join tableB on tableA.col1 =
tableB.col3 where tableB.col3 IS NULL.

The left outer join will force the use of the index and also will pad all the nonmatching columns with NULL values.

Large Insert, Update, and Delete SQLs

Avoid writing unplanned Insert, Update, Delete SQLs (DML) that can potentially affect a large number of records, especially if the production volume is unknown, for example, a statement like

update tableA set column = value
where col 3 = 119

It may not be feasible to predict the number of records that will be affected due to this query, which may lead to system/database resource allocation problems like tempdb. Also, all the records affected by the statement will be locked until the commit is issued. Hence, processing the records through cursor and committing the records frequently based on some commit frequency is suggested.

Do Not Use Positional Numbers

In Order By and Group By conditions, you can use positional numbers of the select statement, such as,

select col1, col2, col3
from tableA
order by 1,2.

Doing so is unwise because another developer may change the Select condition to add a column between col2 and col3 without noticing the Order By, producing unwanted results. Hence, always use the specific column names in the Order By/Group By conditions.

Select col1, col2, col3
from tableA
order by col1, col2, col3

This practice also increases the readability of the query.

Use Group Functions Carefully

When using group functions like MAX, MIN, and AVG, restrict the number of records processed through the WHERE condition. Trying to do group operations on a large table for all the records will eat up system resources. Note that group operations are sort intensive.

Avoid conditions in the HAVING clause. Instead try to have conditions in the WHERE clause. For example, the query

select TaskID, sum(qty)
from Task_Dtl
group by TaskID
having TaskID  > 1000

will do a group operation for all the records in TASK_Dtl, then eliminate Task_Dtl records that are not > 1000. Instead, the SQL can be written as

select TaskID, sum(qty)
from Task_Dtl
where TaskID > 1000
group by TaskID

Use HAVING only to restrict the result set based on the grouped, computed columns.


Summary

In this chapter we provided an insight into how to troubleshoot SQL performance Server and we also looked into some adviable practices for index creation and writing T-SQL code. Chapter 5 provides a detailed overview of the SQL Server Profiler, a very handy tool in debugging SQL Server and application performance issues.


Chapter 24. SQL Server 2000 Service Pack 1

Service Pack 1 (SP1) for Microsoft SQL Server 2000 is provided in three parts.

  • Database Components SP1 provides updates for the database components of an instance of SQL Server 2000, excluding instances of the SQL Server 2000 Desktop Engine. Database Components SP1 includes upgrades to (a) the data engine, (b) database client tools and utilities such as SQL Server Enterprise Manager and osql, and (c) database client connectivity components, such as the Microsoft OLE DB Provider for SQL Server 2000, the SQL Server 2000 ODBC driver, and the client Net-Libraries.

  • Analysis Services SP1 provides updates for the SQL Server 2000 Analysis Services components of a SQL Server 2000 installation, including (a) Analysis Services, Analysis Services client components, such as Analysis Manager and the Microsoft OLE DB Provider for Analysis Services, and database client connectivity components, such as the Microsoft OLE DB Provider for SQL Server 2000, the SQL Server 2000 ODBC driver, and the client Net-Libraries.

  • Desktop Engine SP1 provides updates for the database components of an instance of the SQL Server 2000 Desktop Engine, including (a) the data engine and database client connectivity components, such as the Microsoft OLE DB Provider for SQL Server 2000, the SQL Server 2000 ODBC driver, and the client Net-Libraries.

These three parts of SP1 can be applied individually:

  • SQL Server 2000 sites can use Database Components SP1 to upgrade their database components without upgrading their Analysis Services components or instances of the SQL Server 2000 Desktop Engine.

  • SQL Server 2000 sites also can use Analysis Services SP1 to upgrade their Analysis Services components without upgrading instances of the Desktop Engine or their database components.

  • Desktop Engine sites can use Desktop Engine SP1 to upgrade instances of the Desktop Engine without upgrading Analysis Services or instances of other editions of SQL Server 2000.

However, to upgrade both database components and Analysis Services components, you must apply both Database Components SP1 and Analysis Services SP1.

You can get the service pack on a CD if you subscribe to MSDN, or you can go to Microsoft's Web site and download it:

http://www.microsoft.com/sql/downloads/2000/sp1.asp

Service Pack 1 is available in three self-extracting files from this Web site:

  • SQL2KSP1.exe: Database Components SP1

  • SQL2KASP1.exe: Analysis Services SP1

  • SQL2KDeskSP1.exe: Desktop Engine SP1

The self-extracting files are used to build a set of directories and files on your computer that are the same as those present on the SP1 CD. If you download SP1 you must extract the files to build the service pack directories before you can run SQL Server Setup. The setup process is the same for both the SP1 CD and the extracted SP1 files.


Before the Install

If you are not sure what service pack you're on currently, you can open Query Analyzer and execute the command SELECT @@VERSION to determine it. It will be either 8.00.194—Base SQL Server 2000 or 8.00.384—Service Pack 1 for SQL Server. To determine the version of Analysis Services you're running, simply open Analysis Manager and select About Analysis Services under the Help menu. Your version will be either 8.0.194 or 8.0.382 Analysis Services SP1.

Before installing, extract the setup file to a directory without any spaces and back up your databases. If you're installing the Analysis Services service pack, back up all the cubes as well as your registry. Proper backups are very important because rolling back is not easy; I cannot emphasize backups enough. If you're installing the service pack in a replicated system, make sure that you install the service pack on the Distributor first, then on the Publisher and any Subscribers. Also before you begin the setup, make sure all services related to SQL Server are stopped, including the Microsoft Component Services, Microsoft Message Queuing, and Microsoft COM Transaction Integrator services, if you have than on your machine. Also make sure that your control panel is closed.

If you have applied any hot fixes, ensure that they were wrapped into this service pack. Otherwise, you risk overwriting the critical DLL file(s) that fixed one of your problems. Most of the SQL Server executables and DLL files are replaced during this service pack install, and you could cause regressions by not being careful.

Database Components SP1 Setup automatically detects which edition of SQL Server 2000 is present on the instance of SQL Server 2000 being upgraded, and it upgrades only the components that have been installed for that instance. For example, when the service pack is applied to a computer running SQL Server 2000 Standard Edition, it will not attempt to upgrade components that ship only with SQL Server 2000 Enterprise Edition.

Database Components SP1 can be applied to a single default instance or a named instance of SQL Server. If multiple instances of SQL Server 2000 need to be upgraded to SP1, you must apply SP1 to each instance. When one instance on a computer with one or more instances of SQL Server 2000 is upgraded to SP1, all the tools are upgraded to SP1. There are not separate copies of the tools for each instance on a computer.


Installing Service Pack 1

Installation of the service pack should be easy. If you are performing multiple installs, I recommend creating an unattended installation script, which allows you to install it rapidly and without interaction. After you extract the files, you can run the setup file called setupsql.exe in the \x86\setup folder.

In the first few screens you'll be asked which instance you'd like to install the service pack on and if you'd like to install it on a virtual server. In these two questions, you can see some major upgrades to SQL Server 2000. First, the service pack is instance aware. More important, the service pack is cluster aware. People who have installed SQL Server service packs in the past know what a blessing this is. If you install the service pack on a clustered server, make sure that all the nodes participating in the cluster are online. Second, make sure you install the service pack on the server that owns the SQL Server resources at the time of install. A reboot may be required, so prepare a maintenance window. Since the service pack is cluster aware, you'll have to install it only once on each virtual server in your cluster. Active/Active clusters require two installations since there are two instances running. As you install the service pack, the binaries are automatically copied to each instance.

Next, you are required to provide a sysadmin account to the server if you are using Windows authentication. If you'd like to use SQL Server authentication, you can provide only the sa account password. On Windows 98 and ME machines, you'll have only SQL Server authentication available. If SQL Server is not started already, it will be started at this point.

The first step is the MDAC upgrade, which takes only a moment. The next step is copying the setup program files. The last and longest step is where various scripts are run to upgrade your system catalog. This is why it is so important to have a backup. If the setup breaks anywhere in the installation, your SQL Server may be left in an unstable state because your system catalog may only be half upgraded or only some of the files may have been copied. If problems persist with the installation, do not leave the server in this state. Instead, roll back the service pack using the instructions provided later in this chapter. After the installation completes, you will be asked to back up your master and msdb databases because these databases have been upgraded during the install. You may be required to reboot the system after the service pack install. I recommend rebooting even if you're not asked to since you will be assured that all the services you stopped earlier are correctly started.


Install Database Components SP1

Run the Setup.bat script from either of these locations: the directory containing the extracted service pack files from SQL2KSP1.exe or the service pack directory on the SQL Server 2000 SP1 CD. Setup.bat opens a Setup dialog box that prompts you for information, such as whether you want to use SQL Server Authentication or Windows Authentication. If you choose SQL Server Authentication, you must supply the Setup program with the password for the sa login. If you choose Windows Authentication, you must be running the Setup program while logged on to Windows using a Windows login account. This login account must be a part of the sysadmin fixed server role for the instance of SQL Server 2000 or Desktop Engine you are upgrading.

The Setup program then replaces the existing SQL Server 2000 or Desktop Engine files with the SP1 files, runs several Transact-SQL script files to update system stored procedures, and displays an Authentication Mode dialog box if it detects that the installation is using Mixed Mode authentication with a blank password for the sa login. Leaving the sa login password blank provides users with easy administrative access to SQL Server or Desktop Engine; protect your systems by enforcing an sa password or by using Windows Authentication.

The Authentication Mode dialog box does not default to the current settings for the installation.

On computers running Microsoft Windows 98 and Microsoft Windows Millennium Edition, the Authentication Mode dialog box defaults to Mixed Mode authentication (the only authentication mode supported on these operating systems) and requests that you specify a password for the sa login. If you specify a password, the Setup program will change the sa login password. If you leave the password blank, the Setup program will connect and not change the password.

On computers running Windows NT 4.0 or Microsoft Windows 2000, the Setup program defaults to Windows authentication. Use the dialog box to switch to Windows Authentication Mode or Mixed Mode with an sa login password that is not blank.

Before changing the authentication mode or the password for the sa login, make sure that this change will not affect existing applications. For example, if you change an SQL Server from Mixed Mode authentication to only Windows authentication, existing applications attempting to connect using SQL Server authentication will not be able to connect until the authentication mode is changed to use Windows authentication. Also, if you change the sa login password, applications or administrative processes using the old password will not be able to connect until they have been changed to use the new password.

The Setup program places a record of the actions it performed in the Sqlsp.log file in the Windows directory of the computer on which it was run.

Install Analysis Services SP1

To install Analysis Services SP1, run Setup.exe from either the Msolap\Install subdirectory in the directory containing the extracted Analysis Services SP1 file from SQL2KASP1.exe or the Msolap\Install directory on the SP1 CD. Setup.exe opens a Setup dialog box that prompts you for information and then completes the installation.

Install Desktop Engine SP1

To upgrade an existing installation of the Desktop Engine, run Setup.exe from either the MSDE subdirectory that is in the directory containing the extracted service pack files from SQL2KDeskSP1.exe or the MSDE subdirectory on the SQL Server 2000 SP1 CD. You must apply the correct patch file to each instance of the Desktop Engine you are upgrading. A patch file is a file with an .msp extension that is used to apply this service pack. If you know what installation package was used to install the Desktop Engine, specify a patch file from the command prompt by using the /p option. If you do not know what installation package was used, type SQLRUN after the /p option. With the SQLRUN keyword, the Setup program will choose the correct patch file if all of the .msp files are located in a folder named "Setup" directly under the folder containing Setup.exe and if the instance name of the target Desktop Engine installation is specified on the command prompt if it is not the default instance. The following example will run the Setup program as a patch installation on a named instance called MyInstance:

Setup.exe /p SQLRUN INSTANCENAME=MyInstance


Removing Service Pack 1/Rolling Back to the Previous State

When the service pack is installed, it makes changes to the system tables for maintenance reasons. Due to these changes, SP1 cannot be removed easily. To revert to the build you were running before you installed SP1, you must first uninstall SQL Server 2000 and then reinstall SQL Server 2000. Before you uninstall SQL Server 2000, detach your databases. Reattach them after you reinstall SQL Server 2000. After you have reinstalled SQL Server 2000, you must recreate the necessary logins, recreate scheduled tasks dependent on information in the msdb database, redo any changes you have made in the model database, and reconfigure replication, if you use replication.


Fixing Bugs

Every service pack has standard flavors of bug fixes. For example, every pack has bug fixes for the OLE automation stored procedures (sp_OA sprocs) because there are problems when creating ADO objects with OLE automation stored procedures.

A number of SQL Server 2000 memory leak and access violation (AV) bugs are fixed. In this service pack one of the bug fixes corrects a problem where incomplete searches cause a memory leak in the MSSEARCH service. Another bug the service pack fixes is the CPU being pegged at 100% if the BUILTIN\Administrators login is deleted. Most properly secured systems will have this login id removed.

If you're using DTS regularly, then the service pack is a must-install. You undoubtedly noticed a number of problems in the DTS base install. One is that when you receive a large file using the FTP task in DTS, DTS spends the time to download the file, then deletes it on the destination after it completes the download. This one bug made me stop using the FTP task in SQL Server 2000. There are additional bugs on SMP machines that cause AV errors when you run DTSRUN.EXE, which is very common. Keep in mind that in DTS, you must run the service pack on each workstation running DTS, including the server and your workstation if you're designing a package remotely. Along these same lines there are tool fixes and major XML issues resolved in the service pack.

Fixes for MDAC

Fixes can essentially be divided into two categories, fixes for MDAC, and fixes for SQL Server 2000.

MDAC, as you know, is a part of the SQL Server install itself and hence the service pack for MDAC 2.6 is also a part of Service Pack 1 for SQL Server 2000. If you just want to install the service pack for MDAC, you can do so either by downloading the service pack for MDAC from http://www.microsoft.com/data or by running just the MDAC setup from the SQL Server service Pack 1.An entire list of the service pack fixes can be obtained from the Microsoft's Web site:

http://support.microsoft.com/support/kb/articles/Q300/6/34.ASP

Here, we discuss only some of the ones that are important from the database perspective.

SQL Server ODBC Driver ignores authentication setting

When you use the SQL Server ODBC Driver version 2000.080.0194, which was released with SQL Server 2000, and Microsoft Data Access Components (MDAC) 2.6 RTM, the authentication settings for ODBC data source names (DSNs) are ignored. The driver attempts to log directly into SQL Server using NTLM authentication (integrated security) without prompting for a user ID and password.

This can cause authentication failures, messages that database objects are not available, or unintentional modification of the wrong objects when copies of objects are maintained in multiple databases. The SQL Server ODBC driver allows you to configure datasources to use either Microsoft Windows NT authentication (integrated security) or SQL Server authentication to log into the database. The 2000.080.0194 version of the driver ignores the authentication specified in the data source name (DSN) and attempts to log into the database with NT authentication.

If the Windows NT account does not have permission to log on to the server, authentication errors are thrown. If the account has login permissions but a different database is specified as the default database, "invalid object" errors can be thrown. If multiple copies of objects are kept in separate databases, unintentional modification of the wrong objects can occur.

This problem can cause unexpected behavior in numerous applications. For example, when you use Microsoft Access to link or query tables in an SQL Server database, no login dialog box is presented (because integrated security does not prompt for a login), and tables from the default database for the NT account are presented. No opportunity is given to change the database or login. When you use Microsoft Project with a project that has been saved to an SQL Server database, a connection error is thrown followed by a dialog box that requires the user to clear the Use Trusted Connection checkbox and enter a valid user ID and password.

MDAC 2.6 SQLOLEDB Provider running on Windows 2000 returns no error text description

Installing Microsoft Data Access Components MDAC 2.6 RTM on Windows 2000 or Windows NT 4.0 Professional, Server, or Advanced Server computers prevents error text description from being returned from the Microsoft OLE DB Provider for SQL Server (SQLOLEDB). The Microsoft OLE DB Provider for ODBC (MSDASQL) is not affected. Localization issues cause this bug. If the system locale settings are not set to match the user locale setting for non-United States locale settings, the SQLOLEDB Provider returns an error when it encounters one, but the description text associated with the error is not returned. This is obviously an important fix if you have clients in countries other than the United States.

SQL Server ODBC Driver reuses incorrect parameter length

Under some circumstances, the SQL Server 2000/MDAC 2.6 RTM version of Microsoft ODBC Driver for SQL Server (2000.80.194) reuses a parameter binding that is too short, which truncates the parameter value and causes incorrect or no data to be returned. When a statement is prepared and run by the ODBC driver, a query plan for the statement is cached on the server and a handle for the plan is returned to the driver. The information in the query plan is based partly on the size of the parameters that are being used in the statement. If the size of the bound parameters change, logic in the driver checks to see if the query plan that was cached for the previous running of the statement can still be used. If the query plan cannot be reused because of increased parameter sizes, the plan is unprepared and then reprepared with larger parameter sizes. The driver is incorrectly determining that the older query plan (with the shorter parameter length) can be reused. This causes the parameter values to be truncated, which causes incorrect results. This behavior occurs only when you are using the SQL 2000/MDAC 2.6 ODBC Driver against a SQL 2000 database; it does not occur when you are connecting to an SQL 7.0 Server or with earlier versions of the driver.

SQL OLEDB Provider incorrectly leaves transaction open

Running a certain sequence of commands involving explicit transactions and parameters in an application program using the SQLOLEDB Provider incorrectly leaves an uncommitted transaction open. This can cause severe blocking and other problems. Although the SET option implicit_transactions is set to OFF, the SQLOLEDB Provider is opening an implicit transaction. Through tracing the application, you can see in SQL Server Profiler that the SQLOLEDB Provider is sending the following statements to SQL Server:

SET FMTONLY ON select ... SET FMTONLY OFF
set implicit_transactions off
SET NO_BROWSETABLE OFF

This causes an implicit_transaction to be opened. When using the OLEDB Provider for ODBC and the SQL Server driver instead, the exact same Visual Basic/ADO code sends the following statements to SQL Server:

set implicit_transactions off
SET NO_BROWSETABLE OFF
SET FMTONLY ON select  ... SET FMTONLY OFF

Thus the FMTONLY SELECT opens an implicit transaction with SQLOLEDB. However, because the application never opened this transaction, the application will normally never close it. This transaction will be closed only on the next explicit transaction.

Aggregating ADO Recordsets or Commands may cause memory leak

If a Visual C++ application or dynamic-link library (DLL) aggregates ActiveX Data Objects (ADO) Recordset or Command objects, a significant memory leak can occur due to resources that are not freed when the Release function is called on the object.

This occurs in all versions of ADO prior to and including version 2.6 RTM (2.60.6526). Aggregation in Recordset and Command objects is implemented by using the CComAggObject classes in the Active Template Library (ATL). CComAggObject deletes itself when its reference count goes to zero and simply deletes the Recordset or Command object as well. The Recordset or Command object's Term function is never called, which is where the cleanup of allocated resources occurs. This behavior is likely to be seen only when the application has been written in Visual C++ and when the developer deliberately aggregates Recordset or Command objects. This behavior does not occur in Microsoft Visual Basic applications, because aggregation is not possible from Visual Basic.

SQL Server ODBC Driver generates error when executing a stored procedure on SQL Server 2000

An application that uses the open database connectivity (ODBC) application programming interface (API) to call a SQL Server 2000 stored procedure that has a return value with the SQLExecDirect() API function and open a cursor on the resulting table may return the following errors:

szSqlState = "42000", *pfNativeError = 170, *pcbErrorMsg = 81 MessageText =
"[Microsoft][ODBC SQL Server Driver][SQL Server]Line 1: Incorrect syntax near '?'."
szSqlState = "42000", *pfNativeError = 16945, *pcbErrorMsg = 75 MessageText =
"[Microsoft][ODBC SQL Server Driver][SQL Server]The cursor was not declared."

Before the SQL Server ODBC driver sends the command to the SQL Server, it restructures the ODBC call syntax to the SQL Server sp_opencursor command syntax. In the process of restructuring the command, the driver does not put a space after the name of the stored procedure. For example, calling this stored procedure causes the SQL Server ODBC driver to generate the following sp_cursoropen call table schema and stored procedure.

create table mytable
(f1 integer)
go
create proc myproc
@p1 integer
as
select * from mytable where f1 = @p1
return 2
ODBC Call Syntax
{? = call myproc(?)}
Resulting Call to SQL Server 2000
exec sp_cursoropen @P1 output, N' EXEC @P1=myproc@P2? ', @P2 output, @P3
output, @P4 output, N'@P1 int OUTPUT,@P2 int', @P5 output, 1

Note that the second parameter and the name of the stored procedure do not have a space between them; this causes the syntax error.

SQL Server ODBC Driver does not transfer return values from stored procedures

When running a stored procedure with the SQL Server ODBC Driver that ships with Microsoft Data Access Components (MDAC) version 2.6, you may receive the following error message:

[Microsoft][ODBC SQL Server Driver][SQL Server]Line 1: Incorrect syntax near '?'

In addition to this syntax error, the driver does not populate return values from stored procedures. This bug affects only stored procedures that are opened using any type of server-side cursor (Keyset, Static, or Dynamic). The default Forward-Only, Read-Only cursor does not exhibit this problems. The problem occurs only when you call the SQLExecDirect ODBC function, directly or indirectly, to execute the stored procedure. The problem does not occur if you call SQLPrepare to prepare the statement before calling SQLExecute. This is a bug in the SQL Server ODBC driver that ships with MDAC 2.6.

Access violation occurs when using MSXML in a multithreaded environment

When you use the Microsoft XML Parser (MSXML) versions 2.0, 2.5 or 2.6 in a multithreaded environment, your program may experience an access violation. Depending on the host application the exact symptoms may vary; for example, in a Microsoft Transaction Server (MTS) or COM+ application, an event may be logged indicating that a user component raised an exception. This is due to known issues with MSXML versions 2.0, 2.5 and 2.6.

ADO disconnected recordset that uses parameterized query is not disconnected by SQL Server

The Microsoft ActiveX Data Objects (ADO) technology allows users to obtain a recordset from a data source, "disconnecting" the recordset from its source by setting the Recordset of the ActiveConnection property object to Nothing and then (optionally) closing the corresponding Connection object. Microsoft highly recommends that you use this feature, because it conserves scarce server resources. However, when Microsoft SQL Server is the data source and the recordset is based on a parameterized ADO Command, SQL Server does not disconnect the user.

You can observe this behavior when you use an ADO Command object as the source of your recordset, whether the ADO Command object is based on a parameterized stored procedure or the CommandText property of the ADO Command object is an SQL statement that includes parameters. This behavior occurs when you use either Microsoft OLE-DB Provider for SQL Server (SQLOLEDB) or Microsoft OLE-DB Provider for ODBC drivers (MSDASQL) with the SQL Server ODBC Driver. This behavior does not occur if you use only a Recordset object and call a parameterized stored procedure with an EXECUTE or CALL statement as the Source argument, for example, "EXEC MyProc 1" or "{CALL MyProc(1)} ."

SQL Server 2000 Fixes

For a complete list of SQL Server 2000 fixes, visit the Microsoft Web site:

http://support.microsoft.com/support/kb/articles/Q290/2/12.ASP

Here, we discuss a few important key fixes.

DTS package reports success when FTP tasks doesn't transfer all files

If the FTP task step of a Data Transformation Services (DTS) package is disturbed by a network error, the process may discard the partially downloaded file, continue execution to completion, and report that it "successfully completed execution of package."Because no error is logged in the package error log, the operator may be unaware that files are missing. The corrected behavior is that the DTS package completes execution and displays a package error window with the following message:

Package failed because step % failed

Note that the fix does not attempt to transfer files again; in the event of a network error, you may still find files missing.

CREATE for existing object with IDENTITY column causes duplicate IDENTITY values

When a CREATE statement fails because the object already exists and one of the columns has the IDENTITY property, duplicate IDENTITY values may result. If there is a unique index or primary key on the table, subsequent inserts will fail with a "duplicate key" message. The IDENTITY value is incorrectly set when a CREATE statement fails due to the object already existing.

Incorrect behavior occurs in SQL Server Profiler event generation

Starting in SQL Server 2000, SQL:StmtStarting and SQL:StmtCompleted events display not only the execution of a stored procedure but all the statements within the stored procedure. Due to this change in behavior, the capability of controlling the granularity of information being captured is lost. This behavior has been changed in SQL Server 2000 Service Pack 1—the SQL Server 7.0 behavior is restored. You can now capture the individual statements within a stored procedure by using SP:StmtStarting and SP:StmtCompleted events because the statements are no longer mapped to SQL:StmtStarting and SQL:StmtCompleted events.

DBCC CHECKTABLE may fail if corruption exists in indexes

If you run DBCC CHECKTABLE on a table that has corruption in the indexes, the check may terminate with this error message:

[Microsoft][ODBC SQL Server Driver][Named Pipes]ConnectionCheckForData
(PeekNamedPipe()).
[Microsoft][ODBC SQL Server Driver][Named Pipes]Connection broken.

Note

Running with NOINDEX does not cause the error to occur or terminate the connection.

Update with CASE statements that uses the NEWID() function causes Errors 8951, 8952, 8955 and 8956

An UPDATE statement that modifies data by using a CASE statement and the NEWID function may report one of the following error messages when running a DBCC CHECKTABLE function on the underlying table:

Server: Msg 8951, Level 16, State 1, Line 2 Table Corrupt: Table '#t__00000000000C'
(ID 165575628). Missing or invalid key in index 'main' (ID 2) for the row:
Server: Msg 8955, Level 16, State 1, Line 2 Data row (1:87:0) identified by (RID =
(1:87:0) ) has index values (col1 = 9EF944CD-CBB4-4E7E-A1A4-A2E5B1565FC1 •a ,).
Server: Msg 8952, Level 16, State 1, Line 2 Table Corrupt: Database 'tempdb', index
'#t__00000000000C.main' (ID 165575628) (index ID 2). Extra or invalid key for the
keys:
Server: Msg 8956, Level 16, State 1, Line 2 Index row (1:104:0) with values (col1 =
019973DC-D2E6-4840-AEB8-25DC262A1840 •a ,) points to the data row identified by (RID
= (1:87:0)).

These error messages may occur only if (a) the table has a data type of uniqueidentifier on one of the columns, (b) the preceding column is updated with a CASE statement along with the NEWID function, and (c) there is a unique nonclustered index on the column with the uniqueidentifier datatype.

SELECT may not return all rows if it contains a large number of values in an IN clause on a NUMERIC column

A SELECT statement may not return all the rows that satisfy a WHERE clause if (a) there are a large number of values in the IN clause of that SELECT statement, (b) the datatype of the column used in the IN clause is NUMERIC, and (c) if the execution plan for the SELECT statement shows a "Constant Scan."

Data transfer using ODBC driver for SQL Server fails

When you use the Microsoft ODBC Driver for SQL Server as either the source connection driver or the destination connection driver with the SQL Server Import/Export Wizard, and you choose the Copy Objects and Data Between SQL Server Databases option, you may receive the following error message:

Unable to connect to source for transfer.

[Microsoft][ODBC SQL Server Driver][SQL Server]Login Failed for user ' '.

This error message is caused by incorrectly receiving the connection information from an SQL Server ODBC connection. The fix provided by SQL Server 2000 U.S. Service Pack 1 disables the Copy Objects and Data Between SQL Server Databases option in the Import/Export Wizard when a connection to SQL Server over ODBC is used.

Lock Monitor uses excessive CPU

An SQL Server server appears to stop responding (hang), and query throughput on the server as measured by the "Batch Requests/sec" counter of the SQL Server:SQL Statistics object in System Monitor (Sysmon) is severely hampered. If you monitor the CPU utilization of threads in the SQL Server process space by using the %Processor Time counter of the Threads object in the System Monitor, one thread exhibits high CPU utilization. All other threads (or the vast majority of threads) appear to be "sleeping" and will "track" at close to 0 (zero) processor utilization.

Examination of the value reported for the cpu column of the sysprocesses table in the master database for the Lock Monitor server process id (SPID), SPID 4, shows that the Lock Monitor has used a large amount of CPU time and is continuing to accumulate CPU time as demonstrated by successive queries to the sysprocesses table. Additionally, many SPIDs in the sysprocesses table may be be blocked waiting for some type of lock resource. SQL Server 2000 deadlock detection was enhanced to detect thread-level deadlocks. In some instances, the thread deadlock detection is overly aggressive, and may lead to excessive CPU consumption by the Lock Monitor thread.

Correlated subquery containing UNION ALL may cause access violation

If a correlated subquery uses the UNION ALL clause, SQL Server may produce an access violation (AV) and stack trace.

OpenXML may leak work tables in memory

When you use OPENXML to expose XML data directly to Transact-SQL manipulation, the following error message may appear in your application error log:

clean_tables_xact: active sdes for tabid -170425569
DBCC CHECKDB may cause access violation when the database is in use

The DBCC CHECKDB command may produce an access violation (AV) when run on a database while other users are actively accessing the database. The access violation causes the connection from which DBCC CHECKDB is run to be broken, but no harm is done to the database. The command may be safely restarted, preferably at a time of reduced work load.

Sysprocesses is incorrectly showing Rollback status even though the transaction is not rolling back

Under normal processing conditions, sysprocesses output may show system process IDs (SPIDs) with a status of Rollback when the transaction is not truly being rolled back but is, in fact, being committed. You can see this Rollback status when the CMD column indicates INSERT, UPDATE, or COMMIT TRAN.

DTS scheduled job does not complete after first run

When a scheduled job runs a Data Transformation Services (DTS) package that instantiates a Windows Scripting Host shell object, the Dtsrun.exe program may stop responding (hang). The actual package logic completes, but the Dtsrun.exe process does not complete.

LOCK_TIMEOUT causes transaction to roll back and @@error does not capture error 1222

In the original release of SQL Server 2000, design changes to the software changed the behavior of the LOCK_TIMEOUT setting. A system server process ID (SPID) that incurs a lock timeout is treated like a deadlock victim: The entire batch is rolled back and aborted, and you are unable to check for @@error = 1222. Starting with the release listed in the Resolution section, the software has been changed to revert to the SQL Server 7.0 behavior of aborting only a single statement. This, I believe, is one of the biggest fixes in the service pack. Our application began to fail because of this error, and even though a hot fix was available from Microsoft immediately, we decided to wait for the service pack since hot fixes do not under go regression testing and service packs do; that hot fix had also modified other common files.

Analysis Services: Drillthrough fails if dimension table has an alias in the cube

If your dimension tables are aliased (obtained by right-clicking the title bar of a dimension table in the Cube Editor and selecting Change Alias) and you select columns from the alias for the drillthrough results, the drillthrough may fail. Depending on the data source you are using (Access or SQL Server), an error message that resembles one of the following may be seen:

Unable to drill through. Data source provider error. Statement(s) could not be
prepared.;42000; .... Unable to drill through. Data source provider error.
[Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 12.;07001

The SQL statement that is generated upon drillthrough seems to reference the alias names without generating the DimTable AS Alias clause, and as a result, the statement fails. For example, the following statement may be generated:

SELECT Alias.column from Alias
instead of:
SELECT Alias.column from DimTable AS Alias
Timeout error occurs with large dimensions in dimension editor or wizard

When you complete the design of a large dimension in the Dimension Wizard while the "Count level members automatically" option is selected, you may receive the following error message:

Unable to count the members of the 'XYZ'level. Unable to open the record
set. Error: Timeout expired.

The error may also occur in the Dimension Editor. The command timeout in OLAP Manager is set to 30 seconds. If the counting of the members takes longer than 30 seconds, the error message appears.

Analysis Services server fails with query to cube with multiple partitions and distinct count measure

When you query a cube that has multiple partitions, aggregations stored by way of ROLAP or HOLAP, and a distinct count measure, you may receive the following error in the Windows Event Viewer for the Analysis Services service:

Unexpected fatal error occurred. Attempting to restart server.

The client then receives an error message such as the following:

Connection to the server is lost.
Analysis Services cubes restored from archive may show #ERR when browsed

When an Analysis Services database archive is restored and the database contains a cube that uses the HOLAP storage mode for one or more partitions of the cube, the cube may display #ERR for all the data cells in the cube when you view the cube through the cube browser. The fix for this problem is in the archive functionality of Analysis Services. Therefore, archives created with an Analysis Services version prior to Service Pack 1 may still exhibit this behavior when restored on an Analysis Services server with SP1 applied.

Analysis Services DTS task gets timeout error during incremental update

When performing an incremental update on an Analysis Services cube using the Analysis Services DTS Processing Task, the incremental update may fail with a timeout error if the fact table is large. The DTS Package log will show the following error information:

Step Error Source: Microsoft OLE DB Provider for SQL Server Step Error
Description:Timeout expired.  Step Error code: 80040E31 Step Error Help
File: Step Error Help Context ID:1000440

The incremental update process creates a temporary partition to hold the new data for the cube. This temporary partition is then populated with the new data and merged with the existing partition for the cube. During the creation of the temporary partition, the DTS task attempts to get a record count from the fact table for the cube. If this count is not returned within 30 seconds, the connection returns a timeout error and the DTS task is unable to continue.

Because the partition created by the DTS Processing Task is temporary and is removed after being merged with the existing partition, the count of the fact table is not necessary and has been removed from the partition creation sequence.

Analysis Server HTTPS connections are slower than HTTP and TCP/IP connections

When executing queries over an HTTPS (SSL) connection to an Analysis Services 2000 Server, the query performance may be considerably slower than when the same query is executed over an HTTP or native TCP/IP connection to the server. The module that handles HTTPS connections to the Analysis Server calls the WinInet InternetReadFile function for each individual protocol stream object/property, whereas TCP and HTTP connections use buffering when reading data.


Summary

As always, you should test out the service pack on a stand-by server first before applying it to your production servers. Also remember to back up your database(s) before applying the service pack. The fixes discussed in this chapter I think are very useful. Some of the other fixes may or may not be of use to you depending upon your needs. Overall, I find the service pack to be very helpful in resolving my production server woes.


Part III: Comparison with Oracle 9i

Since Oracle is the biggest competitor of SQL Server 2000 and since many companies that use Oracle are either shifting to SQL Server 2000 (because it's economically viable) or are planning to support SQL Server 2000 as well in order to broaden their customer base, we need to know the differences between the two, how we can convert their applications from Oracle to SQL Server and what it would take from the database perspective to do so. In this part we focus on the differences in the architecture of the two leading RDBMs in the software market and how to migrate from Oracle to SQL Server.


Chapter 25. Oracle 9i vs. SQL Server 2000

Oracle has released three versions of Oracle 8i, 8.1.5, 8.1.6, and 8.1.7, and Oracle 9i, which has some great features. Microsoft SQL Server 2000 is the latest release of the SQL Server product. A Service Pack 1 has also been released for the SQL Server 2000. Both are based on the same relational principles, so there are many similarities between the two, but there are striking differences as well.


Database Definition

Oracle provides a number of different components that make up the entire database environment:

  • Background processes, caches, and memory structures

  • System catalog contained in the system tablespace

  • User-defined tablespaces for data and index storage

  • Redo log files to track the changes in the database

  • Control files and the configuration files

An instance in the case of Oracle refers to the memory structures (system global area (SGA), buffer cache, and so on) and the background processes. A database refers to the physical files. Unless you are using the Oracle Parallel Server (OPS), the relationship between an instance and a database is always 1:1 in Oracle. In the case of OPS, multiple instances access the same database files, providing reliability, since Oracle can continue to operate even when the server fails.

So essentially an Oracle Server consists of an Oracle instance and an Oracle database. The instance must be started in order to access the data in the database. Every time an instance is started, an SGA is allocated and Oracle background processes are started. The SGA is a memory area that is used to store database information that is shared by the database processes. The background processes perform functions on the behalf of the invoking process. The user process in Figure 25.1 is the application program that originates the SQL statements; the server process executes the SQL statements sent from the user process. The database files are the operating system files that provide the actual physical storage for database information. Other files are nondatabase files that are used to configure the instance, authenticate privileged users, and recover the database in the event of a disk failure.

Figure 25.1. Oracle architecture

An SQL Server database provides a logical separation of data, applications, and security mechanisms. An SQL Server installation (an instance) can support multiple databases. Applications built using SQL Server can use databases to logically divide business functionality. There can be multiple instances of SQL Server on a single computer. Each instance of SQL Server can have multiple databases.

Each SQL Server database can support filegroups, which provide the ability to distribute the placement of the data physically. An SQL Server filegroup categorizes the operating system files containing data from a single SQL Server database to simplify database administration tasks, such as backing up. A filegroup is a property of an SQL Server database and cannot contain the operating-system files of more than one database, although a single database can contain more than one filegroup. After a database is created, filegroups can be added to the database.

SQL Server also installs the following databases by default:

  • Master database

  • Model database, a template for all newly created user databases.

  • tempdb database, similar to an Oracle temporary tablespace in that it is used for temporary working storage and sort operations. Unlike the Oracle temporary tablespace, SQL Server users can create temporary tables that are automatically dropped when the user logs off.

  • msdb database, which supports the SQL Server Agent and its scheduled jobs, alerts, and replication information.

  • Pubs and Northwind databases are provided as sample databases for training


Database System Catalogs

Each Oracle database runs on one centralized system catalog, or data dictionary, which resides in the SYSTEM tablespace. Each SQL Server 2000 database maintains its own system catalog, which contains information about database objects (tables, indexes, stored procedures, views, triggers, and so on), constraints, users and permissions, user-defined data types, replication definitions, and files used by the database. SQL Server also contains a centralized system catalog in the master database, which contains system catalogs as well as some information about the individual databases, database names and the primary file location for each database, SQL Server login accounts, system messages, database configuration values, remote and/or linked servers, current activity information, and system stored procedures.

Similar to the SYSTEM tablespace in Oracle, the SQL Server master database must be available to access any other database. It is important to protect against failures by backing up the master database after any significant changes are made in the database. Database administrators can also mirror the files that make up the master database.


Physical and Logical Storage Structures

The Oracle RDBMS comprises tablespaces, which in turn comprise datafiles. Tablespace datafiles are formatted into internal units termed blocks. The block size is set by the DBA when the Oracle database is first created. When an object is created in an Oracle tablespace, the user can specify its space in units called extents (initial extent, next extent, min extents, and max extents). If an extent size is not defined explicitly, a default extent is created. An Oracle extent varies in size and must contain a chain of at least five contiguous blocks.

SQL Server uses filegroups at the database level to control the physical placement of tables and indexes. Filegroups are logical containers of one or more files, and data contained within a filegroup is proportionally filled across all files belonging to the filegroup

If filegroups are not defined and used, database objects are placed in a default filegroup that is implicitly defined during the creation of a database. Filegroups allow you to distribute large tables across multiple files to improve I/O throughput, store indexes on different files than their respective tables, again to improve I/O throughput and disk concurrency, store text, ntext, and image columns (large objects) on separate files from the table, place database objects on specific disk spindles, and back up and restore individual tables or sets of tables within a filegroup. SQL Server formats files into internal units called pages. The page size is fixed at 8192 bytes (8 KB). Pages are organized into extents that are fixed in size at eight contiguous pages (64 KB). When a table or index is created in an SQL Server database, it is automatically allocated one page within an extent. As the table or index grows, it is automatically allocated space by SQL Server. This allows for more efficient storage of smaller tables and indexes when compared to allocating an entire extent as in Oracle.


Striping Data

Oracle-type segments are not needed for most SQL Server installations. Instead, SQL Server can distribute, or stripe, data more efficiently with hardware-based RAID or with software-based RAID available through the Windows Disk Management utility or from third parties. With RAID, you can set up striped volumes (stripe sets in Windows NT 4.0) consisting of multiple disk drives that appear as one logical drive. If database files are created on this striped volume, the disk subsystem assumes responsibility for distributing I/O load across multiple disks. It is recommended that administrators spread out the data over multiple physical disks using RAID.

The recommended RAID configuration for SQL Server is RAID 1 (mirroring) or RAID 5 (stripe sets with an extra parity drive for redundancy). RAID 10 (mirroring of striped sets with parity) is also recommended, but is much more expensive than the first two options. Stripe sets work very well to spread out the usually random I/O done on database files.

If RAID is not an option, filegroups are an attractive alternative and provide some of the same benefits available with RAID. Additionally, for very large databases that might span multiple physical RAID arrays, filegroups may be an attractive way to further distribute your I/O across multiple RAID arrays in a controlled fashion.

Transaction log files must be optimized for sequential I/O and must be secured against a single point of failure. Accordingly, RAID 1 (mirroring) is recommended for transaction logs. When migrating, the size of this drive should be at least as large as the sum of the size of the Oracle online redo logs and the Oracle rollback segment tablespace(s). Create one or more log files that take up all the space defined on the logical drive. Unlike data stored in filegroups, transaction log entries are always written sequentially and are not proportionally filled.


Transaction Logs and Automatic Recovery

The Oracle RDBMS performs automatic recovery each time it is started. It verifies that the contents of the tablespace files are coordinated with the contents of the online redo log files. If they are not, Oracle applies the contents of the online redo log files to the tablespace files (rolls forward) and then removes any uncommitted transactions that are found in the rollback segments (rolls back). If Oracle cannot obtain the information it requires from the online redo log files, it consults the archived redo log files

SQL Server 2000 also performs automatic data recovery by checking each database in the system each time it is started. It first checks the master database and then launches threads to recover all the other databases in the system. For each SQL Server database, the automatic recovery mechanism checks the transaction log. If the transaction log contains any uncommitted transactions, the transactions are rolled back. The recovery mechanism then checks the transaction log for committed transactions that have not yet been written out to the database. If it finds any, it performs those transactions again, rolling forward.

Each SQL Server transaction log has the combined functionality of an Oracle rollback segment and an Oracle online redo log. Each database has its own transaction log that records all changes to the database and is shared by all users of that database. When a transaction begins and a data modification occurs, a BEGIN TRANSACTION event (as well as the modification event) is recorded in the log. This event is used during automatic recovery to determine the starting point of a transaction. As each data modification statement is received, the changes are written to the transaction log prior to being written to the database itself.

SQL Server has an automatic checkpoint mechanism that ensures that completed transactions are regularly written from the SQL Server disk cache to the transaction log file. A checkpoint writes any cached page that has been modified since the last checkpoint to the database. Checkpointing these cached pages, known as dirty pages, onto the database, ensures that all completed transactions are written out to disk. This process shortens the time it takes to recover from a system failure, such as a power outage. This setting can be changed by modifying the recovery interval setting by using SQL Server Enterprise Manager or with Transact-SQL (sp_configure system stored procedure).


Networks

Oracle SQL [image] Net supports networked connections between Oracle database servers and their clients. It communicates with the Transparent Network Substrate (TNS) data stream protocol, and allows users to run many different network protocols without writing specialized code.

With SQL Server, Net-Libraries (network libraries) support the networked connections between the clients and the server by using the Tabular Data Stream (TDS) protocol. They enable simultaneous connections from clients running Named Pipes, TCP/IP Sockets, or other Inter-Process Communication (IPC) mechanisms. The SQL Server 2000 CD-ROM includes all client Net-Libraries, so there is no need to acquire them separately.

SQL Server Net-Library options can be changed after installation. The Client Network utility configures the default Net-Library and server connection information for a client running the Windows 2000, Windows NT 4.0, Windows 95, Windows 98, or Windows Millennium (WinMe) operating systems. All ODBC client applications use the same default Net-Library and server connection information, unless it is changed during ODBC data source setup or explicitly coded in the ODBC connection string.


Comparison of Features Provided by the Two RDBMS

Perhaps the biggest difference is the database definition itself, at the beginning of this chapter (see also Figure 25.2). The best way to compare the feature sets of the two RDBMS is in tabular format. Tables 25.1–25.13 present feature set comparisons briefly; in Chapter 26 we look at the feature sets in greater detail when we talk about the migration process from one RDBMS to the other one.

Figure 25.2. Multiple instances

Table 25.1. General structure comparison
Feature | Oracle | SQL Server
Multiple instances, one database | Oracle Parallel Server | Distributed partitioned views provide some support.
Multiple databases per instance | No | Yes
Multiple instances per server | Yes | Yes
Location of the system catalog | Every database contains its own catalog in the system tablespace. | Every database contains the local portion of its catalog; the master database contains shared security information.
Default admin accounts | Sys/system | Sa/dbo
Temporary space for sorts | Temporary tablespace | Tempdb database
Database configuration settings | Settings stored in control files, init.ora, environment settings, Windows Registry | WindowsRegistry

Table 25.2. Differences in the database optimizer
Feature | Oracle | SQL Server
Fixed optimizer | Y-rule-based optimizer | N/A
Statistics driven optimizer | Y-cost-based optimizer mode | Y
Default statistics collection | Manual; histograms require manual identification | Adaptive, including histograms, this can be scheduled as well.
Join methods | Nested loop, sort merge, hash join (hash_join_enabled) | Nested loop, sort merge, hash join
Parallelism | Parallel Query; Mostly manually configured. (Parallel_Adaptive_Multi_User, Parallel_Automatic_Tuning, Parallel_Threads_Per_CPU, Parallel_Min_Servers, Parallel_Max_Servers) | Adaptive to environment and load.
Automatic summary table usage | Materialized views | Indexed views
Partition elimination | EE with partitioning option | EE; constrained union-all views.
Histogram statistics (skew) | Yes | Yes
Star query optimizations | Yes | Yes
Optimizer hint support | Yes | Yes

Table 25.3. Memory features
Feature | Oracle | SQL Server
Buffer Cache | Static (DB_BLOCK_BUFFERS, BUFFER_POOL_KEEP, BUFFER_POOL_RECYCLE ) | Adaptive; it can be set with a static value as well. Pinning of tables in memory through DBCC PINTABLE or sp_tableoption.
Shared SQL area | Static (SHARED_POOL) | Adaptive (procedure cache)
Sort buffers | Static (SORT_AREA_SIZE, SORT_AREA_RETAINED_SIZE) | Adaptive
Log buffers | Static (LOG_BUFFER_SIZE) | Adaptive
Read ahead buffers | Static (DB_FILE_MULTI_BLOCK_READ_COUNT) | Adaptive
Hash join buffers | Static (HASH_AREA_SIZE) | Adaptive
Bitmap merge buffers | Static (BITMAP_MERGE_AREA_SIZE, CREATE_BITMAP_AREA_SIZE) | N/A

Table 25.4. Storage feature comparison
Feature | Oracle | SQL Server
Group of files | Tablespace (I/O not spread evenly among files) | Filegroup (I/O spread evenly among files)
File | Datafile (auto extend, no auto shrink) | File (auto grow and auto shrink)
Extent | Manual (initial, next, pctincrease, optimal, minextents, maxextents) | 64 K (8 x 8 K Pages)
Smallest unit of storage | Variable size block | 8 K Page
Generic object storage terminology | Segment | N/A
Row address | Rowid | N/A
Fast delete and storage deallocation | Truncate | Truncate
Storage options | Initial, next, minextents, maxextents, pctincrease, pctused, pctfree | Index fill factor
Bitmapped extent management | Locally managed tablespace | Default
Full table scan boundary | Scans to high water mark, regardless of data present | Scans to the end of data

Table 25.5. Transaction support features
Feature | Oracle | SQL Server
Locking model | Oracle multiversioning "snapshot" locking model; Partial ANSI isolation-level support | Industry standard locking model similar to IBM DB2, Informix, Sybase; Full ANSI isolation level support
ANSI isolation levels supported | read committed, serializable | read committed, repeatable read, serializable, read uncommitted
TX rollback information | Rollback segments; susceptible to "Snapshot too old" error | Transaction log
Checkpoint | Checkpoint | Checkpoint
Locking granularity | Row level only | Row level; adaptive and escalates to page, table level for performance optimization.
Database logging | Redo logs | Transaction logs
Archiving of logs | Archived logs (LOG_ARCHIVE_DEST) | N/A
Minimal logging mode | No archive log | Truncates log on checkpoint (simple recovery model)
Full logging mode | Archive log | Truncates log when backed up (full recovery model)
Two phase commit | RECO background process for local and remote databases | Cross server: DTC background process; local server: lightweight internal 2PC Mechanism

Table 25.6. Schema objects
Feature | Oracle | SQL Server
Table and index data stored in b-tree | Indexed organized table | Clustered index
Stored procedures | PL/SQL, Java | Transact-SQL (next release will allow .NET languages as well)
Functions | Yes | Yes
Packages | Yes | No
Summary tables | Materialized views | Indexed views
Indexes on functions | Yes | Yes
Inserts into views | Yes | Yes
On delete cascade DRI | Yes | Yes
On update cascade DRI | No | Yes
INSTEAD OF triggers | Yes | Yes
Aliases | Views, synonyms | Views
Number generator | Sequence | Identity column
Colocated tables | Cluster, hash cluster | N/A
Bitmapped indexes | Yes | No
Bit column indexes | No | Yes

Table 25.7. Datatypes
Oracle | SQL Server
Char | Char
Varchar, Varchar2 | Varchar
Clob, Blob, Bfile, Long, RAW, LongRaw | Text, Varbinary, Image
Number | TinyInt, Int, Numeric, Money
Date | Datetime
RowID | Identity (RID)
N/A | Bit, money, uniqueidentifier

Table 25.8. Backup and recovery
Feature | Oracle | SQL Server
Online backups | Yes | Yes
Backup granularity | DB, tablespace, datafile | DB, filegroup, file
Recovery granularity | Recover to time, scn, cancel | Recover to time, lsn, mark
Differential backups | Incremental backups, reads all blocks, only backs up changed blocks | Differential backups, only reads changed pages
Full logging mode | Archival log mode; archive logs are backed up | Default mode, log grows until stored in a separate directory
Minimal logging mode | No archive log mode | Simple recovery model
Parallelism | Across channels | Across devices

Table 25.9. Availability
Feature | Oracle | SQL Server
Shared database failover | Oracle parallel server cluster | N/A
Shared disk failover | Oracle failsafe, 2 nodes | SQL Server clustering, 4 nodes
Standby database | Standby database with read-only capabilities | Standby log shipping database with read-only capabilities, fast failback
Online index maintenance | Online index rebuild | Online index reorganization

Table 25.10. Security
Feature | Oracle | SQL Server
Admin accounts | Sys/system | Sa/dbo
User and roles | Yes | Yes
Object and system privileges | Yes | Yes
Backup encryption option | No | Yes
Network encryption | Advanced networking option required for SQL [image] Net encryption | SSL Netlib encryption built in
Basic integrated OS/DB security | OPS$ Accounts | Machine accounts
Kerberos integration | Advanced networking option required for Kerberos support | Yes

Table 25.11. Admin tools
Feature | Oracle | SQL Server
DBA console | Enterprise Manager | Enterprise Manager
Database agent | Intelligent Agent | SQL Server Agent
Bulk data loading utility | Sqlldr | BCP and BULKINSERT
Bulk data unloading utility | N/A | BCP
Schema-loading utility | Import, Oracle only | DTS, heterogeneous database support
Schema-unloading utility | Export, Oracle only | DTS, heterogeneous database support
Index tuning tool | Optional Index-Tuning Wizard | Standard Index-Tuning Wizard
SQL plan analysis tool | Explain Plan, some graphical support in Enterprise Manager | Graphical in SQL Query Analyzer; also has showplan_text
Workload tracing tools | SQL_trace and tkprof | SQL Profiler
Database consistency checking | Dbverify, offline only; DB_BLOCK_CHECKSUM | DBCC Check database, online and in parallel

Table 25.12. Networking
Oracle | SQL Server
Multiple protocols via SQL [image] Net | Multiple protocols via Netlib
Transparent links to external database (database links) | Transparent links to external databases (using linked servers), any ODBC or OLEDB database
Network configuration utilities (Net8 Easy config, Net8 Assistant, tnsnames.ora, listener.ora, sqlnet.ora) | N/A
Advanced networking option required for SQL | SSL Netlib encryption built in [image] Net encryption
Multiple instances per server | Multiple instances per server

Table 25.13. Replication
Oracle | SQL Server
Snapshot-Full Refresh | Snapshot; also allows updates
Snapshot-Fast Refresh | Transactional replication; slso allows updates
Updatable snapshots/masters | Merge replication
Synchronous (2PC) updates | Snapshot-immediate 2PC, transactional-immediate 2PC
Asynchronous updates | Merge replication, queued updating subscribers
Complex snapshots | DTS replication integration


New Features in Oracle 9i

Oracle 9i also has some great features.

  • UPSERT Statements. Oracle 9i provides "upsert" capabilities with the MERGE SQL statement: update if it exists, insert if it does not. For example, this SQL was copied from an Oracle 9i white paper:

    MERGE INTO products t
USING products_delta s
ON t.prod_id=s.prod_id
WHEN MATCHED THEN
UPDATE SET t.prod_list_price=s.list_ price,
      t.prod_min_price=s.min_price
WHEN NOT MATCHED THEN
INSERT (prod_id,prod_name,..,prod_list_price,
prod_min_price)
VALUES (s.prod_id, s.prod_name, .., s.list_price,
s.min_price);

  • Real Application Clusters. If you have paid attention to any of the hype surrounding the launch of Oracle 9i, you have no doubt heard about Real Application Clusters. In a nutshell, Real Application Clusters are a way to spread an Oracle 9i database across multiple machines in a cluster of servers, with each server extending both the scalability and the availability of the entire cluster.

  • Data Guard. This new feature automatically handles standby databases from creation to maintenance and failover.

  • Flashback Query. This is a feature that allows you to run a query against the database and return results as they would have been produced at an earlier time. Flashback Query uses the same mechanisms as multiversion read consistency.

  • XMLType. This is a new datatype that lets you store native XML documents directly in the database. XMLType eliminates the need to parse the documents coming into and out of the database.

  • Zero Data Loss. With Oracle 9i, you can specify Zero Data Loss, which means that all writes to the local log file are also written to the log file of the standby database. This feature means that the standby database is always in exactly the same state as the active database, so no data is lost in the event of a failure.


Summary

In this chapter, we went over the key differences in the feature sets of Oracle 9i and SQL Server 2000.


Chapter 26. Migrating from Oracle 9i Databases to SQL Server 2000

Nowadays, in order to expand their customer base, companies that have been developing solutions on just the Oracle RDBMS are beginning to support SQL Server as well. In this chapter we go over the areas that you should keep in mind when you are trying to migrate from Oracle to SQL Server. After noticing the differences in the key areas between the two RDBMS (in the previous chapter), we will now take a look at how to migrate applications from Oracle to SQL Server.


Database Security and Roles

To migrate from Oracle applications to SQL Server 2000 adequately, you must understand how SQL Server implements database security and roles.

Database File Encryption

Microsoft Windows 2000 allows users to encrypt files using the Encrypting File System (EFS). SQL Server 2000 can use EFS. The database files can be encrypted, preventing other users from moving, copying, or viewing their contents. Encryption is done on the operating system level, not the logical database level. After SQL Server opens the encrypted file, the data within the file appears as unencrypted.

Network Security

SQL Server 2000 supports the use of the Secure Sockets Layer (SSL) to encrypt network communications between itself and clients. Encryption applies to all intercomputer protocols supported by SQL Server 2000 and is either 40- or 128-bit depending on the version of the Windows operating system on which SQL Server is running.

Login Accounts

A login account allows a user to access SQL Server data or administrative options. The guest login account allows users to log in to SQL Server and view only databases that allow guest access. (The guest account is not set up by default; it must be created.)

A login account allows a user to administer or access data in an instance of SQL Server 2000. SQL Server 2000 uses Windows authentication and SQL Server authentication to authenticate logins. For Windows authentication, a DBA specifies which Windows login accounts can be used to connect to an instance of SQL Server 2000. Users logged in to Windows using these accounts can connect to SQL Server 2000 without having to specify a separate database login and password. When using Windows authentication, SQL Server 2000 uses the security mechanisms of Windows NT 4.0 or Windows 2000 to validate login connections and relies on a user's Windows security credentials. Users do not need to enter login IDs or passwords for SQL Server 2000 because their login information is taken directly from the trusted network connection. This method functions like the IDENTIFIED EXTERNALLY option associated with Oracle user accounts.

For SQL Server authentication, a DBA defines a separate database login account. A user must specify this login account and its password when attempting to connect to SQL Server 2000. The database login is not related to the user's Windows login. This method functions like the IDENTIFIED BY PASSWORD option associated with Oracle user accounts.

Each instance of SQL Server 2000 is run in one of two authentication modes. In the Windows authentication mode (known as integrated security in earlier versions of SQL Server), SQL Server 2000 allows only connections that use Windows authentication. In mixed mode, connections can be made using either Windows authentication or SQL Server authentication.

Groups, Roles, and Permissions

SQL Server and Oracle use permissions to enforce database security. SQL Server statement-level permissions are used to restrict the ability to create new database objects (similar to the Oracle system-level permissions).

SQL Server also offers object-level permissions. As in Oracle, object-level ownership is assigned to the creator of the object and cannot be transferred. Object-level permissions must be granted to other database users before they can access the object. Members of the sysadmin fixed server role, db_owner fixed database role, or db_securityadmin fixed database role can also grant permissions on one user's objects to other users.

SQL Server statement- and object-level permissions can be granted directly to database user accounts. However, it is often easier to administer permissions to database roles. SQL Server roles are used for granting and revoking privileges to groups of database users (much like Oracle roles). Roles are database objects associated with a specific database. There are a few specific fixed server roles associated with each installation, which work across databases. An example of a fixed server role is sysadmin. Windows groups, as well as database users, can be added as SQL Server logins. Permissions can be granted to a Windows group or a Windows user.

A database can have any number of roles or Windows groups. The default role public is always found in every database and cannot be removed. The public role functions much like the PUBLIC account in Oracle. Each database user is always a member of the public role. A database user can be a member of any number of roles in addition to the public role. A Windows user or group can also be a member of any number of roles and is also always in the public role.

Database Users and the Guest Account

In Microsoft SQL Server, a user login account must be authorized to use a database and its objects. One of the following methods can be used by a login account to access a database: The login account can be specified as a database user, or the login account can use a guest account in the database. A Windows group login can be mapped to a database role. Individual Windows accounts that are members of that group can then connect to the database.

Members of the db_owner or db_accessadmin roles or the sysadmin fixed server role create the database user account roles. An account can include several parameters: the SQL Server login ID, database user name (optional), and up to one role name (optional). The database user name does not have to be the same as the user's login ID. If a database user name is not provided, the user's login ID and database user name are identical. If a role name is not provided, the database user is only a member of the public role. After creating the database user, the user can be assigned to as many roles as necessary.

Members of the db_owner or db_accessadmin roles can also create a guest account. The guest account allows any valid SQL Server login account to access a database even without a database user account. By default, the guest account inherits any privileges that have been assigned to the public role; however, these privileges can be changed to be greater or less than that of the public role.

A Windows user account or group account can be granted access to a database, just as an SQL Server login can. When a Windows user who is a member of a group connects to the database, the user receives the permissions assigned to the Windows group. If a member of more than one Windows group that has been granted access to the database connects to the database, the user receives the combined rights of all of the groups to which he or she belongs.

Sysadmin Role

Members of the SQL Server sysadmin fixed server role have similar permissions to that of an Oracle DBA. In SQL Server, the sa SQL Server authentication login account is a member of this role by default, as are members of the local administrators group if SQL Server is installed on a computer running Windows 2000. A member of the sysadmin role can add or remove Windows users and groups, as well as SQL Server logins. Members of this role typically have the following responsibilities:

  • Installing SQL Servers

  • Configuring servers and clients

  • Creating databases

  • Establishing login rights and user permissions

  • Transferring data in and out of SQL Server databases

  • Backing up and restoring databases

  • Implementing and maintaining replication

  • Scheduling unattended operations

  • Monitoring and tuning SQL Server performance

  • Diagnosing system problems

A member of the sysadmin fixed server role can access any database and all the objects (including data) on a particular instance of SQL Server. Similar to an Oracle DBA, several commands and system procedures can be issued only by members of the sysadmin role.

db-owner Role

Although an SQL Server database is similar to an Oracle tablespace in use, it is administered differently. Each SQL Server database is a self-contained administrative domain. Each database is assigned a database owner (dbo). This user is always a member of the db_owner fixed database role. Other users can also be members of the db_owner role. Any user who is a member of this role has the ability to manage the administrative tasks related to her database (unlike Oracle, in which one DBA manages the administrative tasks for all tablespaces). These tasks include the following:

  • Managing database access

  • Changing database options (read-only, single user, and so on)

  • Backing up and restoring the database contents

  • Granting and revoking database permissions

  • Creating and dropping database objects

Members of the db_owner role have permission to do anything within their database. Most rights assigned to this role are separated into several fixed database roles or can be granted to database users. It is not necessary to have sysadmin server-wide privileges to have db_owner privileges in a database.


Defining Database Objects

Oracle database objects (tables, views, and indexes) can be moved to SQL Server easily because each RDBMS closely follows the SQL-92 standard that regards object definitions. Converting Oracle SQL table, index, and view definitions to SQL Server table, index, and view definitions requires relatively simple syntax changes. Table 26.1 shows some differences in database objects between Oracle and SQL Server.

It is assumed that you are starting with an Oracle SQL script or program that is used to create your database objects. Copy this script or program and make the following modifications.

  1. Ensure that database object identifiers comply to Microsoft SQL Server naming conventions. You may need to change only the names of indexes.

  2. Consider the data storage parameters your SQL Server database will require. If you are using RAID, no storage parameters are required.

    Table 26.1. Database objects
    Category | Microsoft SQL Server | Oracle
    Number of columns | 1024 | 1000
    Row size | 8060 bytes, including 16 bytes to point to each text or image column | Unlimited (only one long or long raw allowed per row)
    Maximum number of rows | Unlimited | Unlimited
    Blob-type storage | 16-byte pointer stored with row; data stored on other data pages | One long or long raw per table; must be at end of row; data stored on same block(s) with row
    Clustered table indexes | 1 per table | 1 per table (index-organized tables)
    Nonclustered table indexes | 249 per table | Unlimited
    Maximum number of columns in single index | 16 | 32
    Maximum length of column values within of an index | 900 bytes | 40% of block
    Table naming convention | [[[Server.]database.]owner.] table_name | [schema.]table_name
    View naming convention | [[[Server.]database.]owner.] table_name | [schema.]table_name
    Index naming convention | [[[Server.]database.]owner.] table_name | [schema.]table_name

  3. Modify Oracle constraint definitions to work in SQL Server. If tables cross databases, use triggers to enforce foreign key relationships.

  4. Modify the CREATE INDEX statements to take advantage of clustered indexes.

  5. Use data transformation services (DTS) to create new CREATE TABLE statements. Review the statements, taking note of how Oracle data types are mapped to SQL Server data types.

  6. Remove any CREATE SEQUENCE statements. Replace the use of sequences with IDENTITY or uniqueidentifier columns in CREATE TABLE or ALTER TABLE statements.

  7. Modify CREATE VIEW statements if necessary.

  8. Remove any reference to synonyms.

  9. Evaluate the use of SQL Server temporary tables and their usefulness in your application.

  10. Change any Oracle CREATE TABLE…AS SELECT commands to SQL Server SELECT…INTO statements.

  11. Evaluate the potential use of user-defined rules, data types, and defaults.

Database Object Identifiers

Table 26.2 compares how Oracle and SQL Server handle object identifiers. In most cases, you do not need to change the names of objects when migrating to SQL Server.

Qualifying Table Names

When accessing tables that exist in your Oracle user account, the table can be selected simply by its unqualified name. Accessing tables in other Oracle schemas requires that the schema name be prefixed to the table name with a single period (.). Oracle synonyms can provide additional location transparency.

Table 26.2. Object identifiers
Oracle | Microsoft SQL Server

1–30 characters in length.

Database names: Up to 8 characters long

Database link names: Up to 128 characters long

1–128 Unicode characters in length.

Temporary table names: Up to 116 characters
Identifier names must begin with an alphabetic character and contain alphanumeric characters or the characters _, $, and #. | Identifier names can begin with an alphanumeric character or an _ and can contain virtually any character. If the identifier begins with a space, or contains characters other than _, @, #, or $, [ ] (delimiters) must be used around the identifier name. If an identifier begins with @, it is a local variable; if #, it is a local temporary object; if ##, it is a global temporary object.
Tablespace names must be unique. | Database names must be unique.
Identifier names must be unique within user accounts (schemas) | Identifier names must be unique within database user accounts.
Column names must be unique within tables and views. | Column names must be unique within tables views.
Index names must be unique within a user's schema. | Index names must be unique within database table names.

SQL Server uses a different convention when it references tables. Because one SQL Server login account can create a table by the same name in multiple databases, the following convention is used to access tables and views: [[database_name.]owner_name.]table_name.

Here are guidelines for naming SQL Server tables and views:

  • Using the database name and user name is optional. When a table is referenced only by name, SQL Server searches for the table in the current user's account in the current database. If it does not find one, it looks for an object of the same name owned by the reserved user name of dbo in the database. Table names must be unique within a user's account within a database.

  • The same SQL Server login account can own tables with the same name in multiple databases. For example, the USER account owns the database objects

    USER_DB.USER.USER_TABLE1
OTHER_DB.USER.USER_TABLE1

    The qualifier is the database username, not the SQL Server login name, because they do not have to be the same.

  • At the same time, other users in these databases may own objects by the same name:

    USER_DB.DBO.USER_TABLE1
USER_DB.DEPT_ADMIN.USER_TABLE1
USER_DB.STUDENT_ADMIN.USER_TABLE1
OTHER_DB.DBO.USER_TABLE1

    Therefore, it is recommended that you include the owner name as part of the reference to a database object. If the application has multiple databases, it is recommended that the database name also be included as part of the reference. If the query spans multiple servers, include the server name.

  • Every connection to SQL Server has a current database context, set at login time with the USE statement. For example, assume the following scenario: A user, using the USER account, is logged in to the USER_DB database. The user requests the USER_TABLE table. SQL Server searches for the table USER.USER_TABLE. If the table is found, SQL Server performs the requested database operation on USER_DB.USER.USER_TABLE. If the table is not found in the USER database account, SQL Server searches for USER_DB.DBO.USER_TABLE in the dbo account for that database. If the table is still not found, SQL Server returns an error message indicating that the table does not exist.

  • If another user, for example DEPT_ADMIN, owns the table, the table name must be prefixed with the database user's name (DEPT_ADMIN.USER_TABLE). Otherwise, the database name defaults to the database that is currently in context.

  • If the referenced table exists in another database, the database name must be used as part of the reference. For example, to access the USER_TABLE table owned by USER in the OTHERDB database, use OTHER_DB.USER.USER_TABLE

The object's owner can be omitted by separating the database and table name by two periods. For example, if an application references USER_DB..USER_TABLE, SQL Server searches as follows:

USER_DB.current_user.USER_TABLE
USER_DB.DBO.USER_TABLE

If the application uses only a single database at a time, omitting the database name from an object reference makes it easy to use the application with another database. All object references implicitly access the database that is currently being used. This is useful when you want to maintain a test database and a production database on the same server.

Creating Tables

Because Oracle and SQL Server support SQL-92 entry-level conventions for identifying RDBMS objects, the CREATE TABLE syntax is similar. Oracle database object names are not case-sensitive. In SQL Server, database object names can be case-sensitive, depending on the installation options selected. If you select a case-sensitive collation, then everything, the data as well as the database objects, becomes case-sensitive.

Oracle Microsoft SQL Server
CREATE TABLE
[schema.]table_name
(
{col_name column_properties
[default_expression] [constraint [constraint
[...constraint]]]| [[,] constraint]}
[[,] {next_col_name | next_constraint} ...]
)
[Oracle Specific Data Storage Parameters]

CREATE TABLE
[server.][database.][owner.] table_name
(
{col_name column_properties[constraint
[constraint [...constraint]]]| [[,] constraint]}
[[,] {next_col_name | next_constraint} ...]
)
[ON file group_name]

When SQL Server is first set up, the default sort order is dictionary order, case-insensitive. (This can be configured differently using SQL Server Setup.) Because Oracle object names are always unique, you should not have any problems moving the database objects to SQL Server. It is recommended that all table and column names in both Oracle and SQL Server be uppercase to avoid problems if a user installs on a case-sensitive SQL Server.

Table and Index Storage Parameters

With Microsoft SQL Server, using RAID usually simplifies the placement of database objects. An SQL Server clustered index is integrated into the structure of the table, like an Oracle index-organized table.

Oracle Microsoft SQL Server
CREATE TABLE ADMIN.DEPT(
DEPT     VARCHAR(4) NOT NULL,
DNAME    VARCHAR(30) NOT NULL,
CONSTRAINT dept_pk PRIMARY KEY (dept)
,
CONSTRAINT dept_dname_unique
UNIQUE(dname)
)
ORGANIZATION INDEX

CREATE TABLE
USER_DB.ADMIN.DEPT (
DEPT     VARCHAR(4) NOT
NULL,
DNAME    VARCHAR(30) NOT
NULL,
CONSTRAINT DEPT_DEPT_PK
                PRIMARY KEY
CLUSTERED (DEPT),
CONSTRAINT DEPT_DNAME_UNIQUE
                UNIQUE
NONCLUSTERED (DNAME)
)

Using Oracle, a table can be created with any valid SELECT command. Microsoft SQL Server provides the same functionality with different syntax.The SELECT…INTO statement creates a new table and populates it with the results of the SELECT statement. Referential integrity definitions are not transferred to the new table. If the database recovery mode is set to FULL, then the SELECT…INTO statement is logged in the transaction log and a point-in-time recovery can take place.

Oracle Microsoft SQL Server
CREATE TABLE MyTable2 AS SELECT *
FROM MyTable

SELECT * INTO MyTable2
FROM MyTable

Views

The syntax used to create views in SQL Server is similar to that of Oracle.

Oracle Microsoft SQL Server
CREATE [OR REPLACE] [FORCE |
NOFORCE] VIEW [schema.]view_name
[(column_name [, column_name]...)]
AS select_statement
[WITH CHECK OPTION [CONSTRAINT
name]]
[WITH READ ONLY]

|CREATE VIEW [owner.]view_name
[(column_name [, column_name]...)]
WITH ENCRYPTION]
AS select_statement [WITH CHECK
 OPTION]

SQL Server views require that the tables exist and that the view owner has privileges to access the requested tables(s) specified in the SELECT statement (similar to the Oracle FORCE option).

By default, data modification statements on views are not checked to determine whether the rows affected are within the scope of the view. To check all modifications, use the WITH CHECK OPTION. The primary difference between the WITH CHECK OPTION is that Oracle defines it as a constraint and SQL Server does not. Otherwise, it functions alike in both.

Oracle provides a WITH READ ONLY option when defining views. SQL Server-based applications can achieve the same result by granting only SELECT permission to the users of the view.

Both SQL Server and Oracle views support derived columns, using arithmetic expressions, functions, and constant expressions. Some of the specific SQL Server differences follow:

  • Data modification statements (INSERT or UPDATE) are allowed on multitable views if the data modification statement affects only one base table. Data modification statements cannot be used on more than one table in a single statement.

  • READTEXT or WRITETEXT cannot be used on text or image columns in views.

  • ORDER BY, COMPUTE, FOR BROWSE, or COMPUTE BY clauses cannot be used.

  • The INTO keyword cannot be used in a view.

When a view is defined with an outer join and is queried with a qualification on a column from the inner table of the outer join, the results from SQL Server and Oracle can differ.


Indexed Views/Materialized Views

Microsoft SQL Server 2000 Enterprise Edition introduces indexed views as discussed in Chapter 10. Like Oracle's materialized views, indexed views are views that physically store their indexed result set on disk. Indexed views are automatically updated when their base data is updated. Indexed views are capable of providing a large boost to performance in decision-support systems in which large amounts of data must be aggregated or in online transaction processing (OLTP) systems in which many joins are used to aggregate slowly changing data.

SQL Server indexed views cannot reference objects in remote databases. SQL Server has multiple internal replication schemes including both Snapshot and Transactional replication that provide more functionality in terms of moving data among servers.

In SQL Server, for a view to be eligible for indexing, it must be defined with the SCHEMABINDING option that locks the underlying table schemas that the view references. The view also cannot reference only nondeterministic functions. The first index created against the view must be a clustered unique index so that SQL Server can quickly locate rows when the data in the base schema is modified.

Indexes

Microsoft SQL Server offers clustered and nonclustered index structures made up of pages that form a branching structure known as a B-tree (similar to the Oracle B-tree index structure). The starting page (root level) specifies ranges of values within the table. Each range on the root-level page points to another page (decision node), which contains a more limited range of values for the table. In turn, these decision nodes can point to other decision nodes, further narrowing the search range. The final level in the branching structure is called the leaf level.

Clustered Indexes

Clustered indexes are implemented in Oracle as index-organized tables. A clustered index is an index that has been physically merged with a table. The table and index share the same storage area. The clustered index physically rearranges the rows of data in indexed order, forming the intermediate decision nodes. The leaf pages of the index contain the actual table data. This architecture permits only one clustered index per table. Microsoft SQL Server automatically creates a clustered index for the table whenever a PRIMARY KEY or UNIQUE constraint is placed on the table. Clustered indexes are useful for the following:

  • Primary keys

  • Columns that are not updated

  • Queries that return a range of values, using operators such as BETWEEN, >, >=, <, and <=, for example,

    SELECT * FROM MyTable WHERE CREATE_DATE_TIME BETWEEN '1/1/00' AND '12/31/01'

  • Queries that return large result sets, such as

    SELECT * FROM USER_MASTER WHERE LNAME = 'SHARMA'

  • Columns that are used in sort operations (ORDER BY, GROUP BY)

  • Distributing update activity in a table to avoid hot spots. Hot spots are often caused by multiple users inserting into a table with an ascending key. This application scenario is usually addressed by row-level locking.

Dropping and recreating a clustered index is a common technique for reorganizing a table in SQL Server. It is an easy way to ensure that data pages are contiguous on disk and to reestablish some free space in the table. It is similar to exporting, dropping, and importing a table in Oracle.

An SQL Server clustered index is not at all like an Oracle cluster. An Oracle cluster is a physical grouping of two or more tables that share the same data blocks and use common columns as a cluster key. SQL Server does not have a structure similar to an Oracle cluster.

An a general rule, defining a clustered index on a table improves SQL Server performance and space management. If you do not know the query or update patterns for a given table, you can create the clustered index on the primary key.

Nonclustered Indexes

In nonclustered indexes, the index data and the table data are physically separate, and the rows in the table are not stored in the order of the index. You can move Oracle index definitions to Microsoft SQL Server nonclustered index definitions. For performance reasons, however, you might want to choose one of the indexes of a given table and create it as a clustered index.

Index Syntax and Naming

In Oracle, an index name is unique within a user account. In SQL Server, an index name must be unique within a table name, but it does not have to be unique within a user account or database. Therefore, when creating or dropping an index in SQL Server, you must specify both the table name and the index name. Additionally, the SQL Server DROP INDEX statement can drop multiple indexes at one time.

Oracle Microsoft SQL Server
CREATE [UNIQUE] INDEX
[schema].index_name
     ON [schema.]table_name
(column_name [, column_name]...)
[INITRANS n]
[MAXTRANS n]
[TABLESPACE tablespace_name]
[STORAGE storage_parameters]
[PCTFREE n]
[NOSORT]
DROP INDEX ABC;

CREATE [UNIQUE] [CLUSTERED |
NONCLUSTERED]
 INDEX index_name ON table (column
[,...n])
[WITH
  [PAD_INDEX]
  [[,] FILLFACTOR = fillfactor]
  [[,] IGNORE_DUP_KEY]
  [[,] DROP_EXISTING]
  [[,] STATISTICS_NORECOMPUTE]
]
[ON file group]
DROP INDEX USER_DB.STUDENT.DEMO_IDX,
USER_DB.GRADE.DEMO_IDX

Index Data Storage Parameters

The FILLFACTOR option in SQL Server functions in much the same way as the PCTFREE variable does in Oracle. As tables grow in size, index pages split to accommodate new data. The index must reorganize itself to accommodate new data values. The fill factor percentage is used only when the index is created, and it is not maintained afterwards.

The FILLFACTOR option (values are 0 through 100) controls how much space is left on an index page when the index is initially created. The default fill factor of 0 is used if none is specified—this will completely fill index leaf pages and leave space on each decision node page for at least one entry (two for nonunique clustered indexes).

A lower fill factor value initially reduces the splitting of index pages and increases the number of levels in the B-tree index structure. A higher fill factor value uses index page space more efficiently, requires fewer disk I/Os to access index data, and reduces the number of levels in the B-tree index structure. The PAD_INDEX option specifies that the fill factor setting be applied to the decision node pages as well as to the data pages in the index. Although it may be necessary to adjust the PCTFREE parameter for optimal performance in Oracle, it is seldom necessary to include the FILLFACTOR option in a CREATE INDEX statement. The fill factor is provided for fine-tuning performance. It is useful only when creating a new index on a table with existing data, and then it is useful only when you can accurately predict future changes in the data.

If you have set PCTFREE to 0 for your Oracle indexes, consider using a fill factor of 100. This fill factor is used when no inserts or updates will occurr in the table (a read-only table). When the fill factor is set to 100, SQL Server creates indexes with each page 100 percent full.

Ignoring Duplicate Keys

With both Oracle and SQL Server, users cannot insert duplicate values for a uniquely indexed column or columns. An attempt to do so generates an error message. Nevertheless, SQL Server lets the developer choose how the INSERT or UPDATE statement will respond to the error.

If IGNORE_DUP_KEY was specified in the CREATE INDEX statement and an INSERT or UPDATE statement that creates a duplicate key is executed, SQL Server issues a warning message and ignores (does not insert) the duplicate row. If IGNORE_DUP_KEY is not specified for the index, SQL Server issues an error message and rolls back the entire INSERT statement. For more information about these options, see SQL Server Books Online.

Indexes on Computed Columns

Oracle allows you to place an index directly on a function. Microsoft SQL Server allows for indexes to be placed on computed columns within a table. Within SQL Server, a table can consist of multiple computed columns but must have at least one noncomputed column. Computed columns can consist of either SQL Server functions or user-defined functions, but the functions must be deterministic in nature. That is, the function must return the same value each time it is called with identical parameters. For example, the SQL Server GETDATE() function is nondeterministic because it is always called with the same parameters and returns a different value each time.

Using Temporary Tables

An Oracle application might have to create tables that exist for short periods. The application must ensure that all tables created for this purpose are dropped at some point. If the application fails to do this, tablespaces can quickly become cluttered and unmanageable.

Microsoft SQL Server provides temporary table database objects, which are created for just such a purpose. These tables are always created in the tempdb database. The table name determines how long they reside within the tempdb database.

#table_name This local temporary table exist only for the duration of a user session or the procedure that created it. It is automatically dropped when the user logs off or the procedure that created the table completes. These tables cannot be shared between multiple users. No other database users can access this table. Permissions cannot be granted or revoked on this table.

##table_name This global temporary table also typically exists for the duration of a user session or procedure that created it. This table can be shared among multiple users. It is automatically dropped when the last user session referencing it disconnects. All other database users can access this table. Permissions cannot be granted or revoked on this table.

Indexes can be defined for temporary tables. Views can be defined only on tables explicitly created in tempdb without the # or ## prefix. When the user exits, the table and index are automatically dropped. You may find that the benefits associated with using temporary tables justify a revision in your program code.


Data Types

Although some data type conversions from Oracle to SQL Server are straightforward, other data type conversions require evaluating a few options. Using the DTS Import/Export Wizard to automate the creation of the new CREATE TABLE statements is recommended. These statements will provide you with the recommended conversion of the data types. We discussed these briefly in Chapter 25; Table 26.3 gives a recommended conversion strategy for the data types.

Using Unicode Data

The Unicode specification defines a single encoding scheme for practically all characters widely used in businesses around the world. All computers consistently translate the bit patterns in Unicode data into characters using the single Unicode specification. This ensures that the same bit pattern is always converted to the same character on all computers. Data can be freely transferred from one database or computer to another without concern that the receiving system will correctly translate the bit patterns into characters.

One problem with data types that use 1 byte to encode each character is that the data type can represent only 256 different characters. This forces multiple encoding specifications (or code pages) for different alphabets. It is also impossible to handle systems such as the Japanese Kanji or Korean Hangul alphabets that have thousands of characters.

Microsoft SQL Server translates the bit patterns in char, varchar, and text columns to characters using the definitions in the code page installed with SQL Server. Client computers use the code page installed with the operating system to interpret character bit patterns.

There are many different code pages. Some characters appear on some code pages but not on others. Some characters are defined with one bit pattern on some code pages and a different bit pattern on other code pages. When you build international systems that must handle different languages, it becomes difficult to pick code pages for all the computers that meet the language requirements of multiple countries. It is also difficult to ensure that every computer performs the correct translations when interfacing with a system that uses a different code page.The Unicode specification addresses this problem by using 2 bytes to encode each character. There are enough different patterns (65,536) in 2 bytes for a single specification covering the most common business languages. Because all Unicode systems consistently use the same bit patterns to represent all characters, there is no problem with characters being converted incorrectly when moving from one system to another. In SQL Server, nchar, nvarchar, and ntext data types support Unicode data.

Table 26.3. Data type conversion strategy
Oracle | Microsoft SQL Server
CHAR | char is recommended. char type columns are accessed somewhat faster than varchar columns because they use a fixed storage length.
VARCHAR2 and LONG | varchar or text. (If the length of the data values in your Oracle column is 8000 bytes or less, use varchar; otherwise, use text.)
RAW and LONG RAW | varbinary or image. (If the length of the data values in your Oracle column is 8000 bytes or less, use varbinary; otherwise, you must use image.)
NUMBER

For an integer between 1 and 255, use tinyint. For an integer between –32,768 and 32,767, use smallint.For an integer between –2,147,483,648 and 2,147,483,647, use int.For an integer between –263 and 263, use bigint. If you require a float-type number, use numeric (has precision and scale).

Note

Do not use float or real, because rounding may occur (Oracle NUMBER and SQL Server numeric do not round). If you are not sure, use numeric; it most closely resembles Oracle NUMBER data type.
DATE | datetime
ROWID | Use the identity column type or the uniqueidentifier data type and the NEWID function.
CURRVAL, NEXTVAL | Use the identity column type, and @@IDENTITY global variable, IDENT_SEED() and IDENT_INCR() functions.
SYSDATE | GETDATE()
USER | USER

Object-Level Permissions

Microsoft SQL Server object privileges can be granted to, denied from, and revoked from other database users, database groups, and the public role. SQL Server does not allow an object owner to grant ALTER TABLE and CREATE INDEX privileges for the object as Oracle does. Those privileges must remain with the object owner.

The GRANT statement creates an entry in the security system that allows a user in the current database to work with data in the current database or to execute specific Transact-SQL statements. The syntax of the GRANT statement is identical in Oracle and SQL Server.

The Transact-SQL DENY statement creates an entry in the security system that denies a permission from a security account in the current database and prevents the security account from inheriting the permission through its group or role memberships. Oracle does not have a DENY statement.

The Transact-SQL REVOKE statement removes a previously granted or denied permission from a user in the current database. In Oracle, the REFERENCES privilege can be granted only to a user. SQL Server allows the REFERENCES privilege to be granted to both database users and database groups. The INSERT, UPDATE, DELETE, and SELECT privileges are granted in the same way in both Oracle and SQL Server.


Enforcing Data Integrity and Business Rules

Enforcing data integrity ensures the quality of the data in the database. Two important steps when planning tables are identifying valid values for a column and deciding how to enforce the integrity of the data in the column. Data integrity falls into four categories, and is enforced in various ways (Table 26.4).

Table 26.4. Data integrity
Integrity type How enforced
Entity integrity

PRIMARY KEY constraint

UNIQUE constraint

IDENTITY property
Domain integrity

Domain DEFAULT definition

FOREIGN KEY constraint

CHECK constraint

Nullability
Referential integrity

FOREIGN KEY constraint

CHECK constraint
User-defined integrity

All column- and table-level constraints in CREATE TABLE

Stored procedures

Triggers

Naming Constraints

You should always name your constraints explicitly. If you do not, Oracle and Microsoft SQL Server will use different naming conventions to name the constraint implicitly. These differences in naming can complicate your migration process unnecessarily. The discrepancy appears when dropping or disabling constraints, because constraints must be dropped by name. The syntax for explicitly naming constraints is the same for Oracle and SQL Server.

Primary Keys and Unique Columns

The SQL-92 standard requires that all values in a primary key be unique and that the column not allow null values. Both Oracle and Microsoft SQL Server enforce uniqueness by automatically creating unique indexes whenever a PRIMARY KEY or UNIQUE constraint is defined. Additionally, primary key columns are automatically defined as NOT NULL. Only one primary key is allowed per table.

An SQL Server clustered index is created by default for a primary key, though a nonclustered index can be requested. The Oracle index on primary keys can be removed by either dropping or disabling the constraint, whereas the SQL Server index can be removed only by dropping the constraint.

In either RDBMS, alternate keys can be defined with a UNIQUE constraint. Multiple UNIQUE constraints can be defined on any table. UNIQUE constraint columns are nullable. In SQL Server, a nonclustered index is created by default unless otherwise specified.When moving your application, it is important to note that SQL Server allows only one row to contain the value NULL for the complete unique key (single or multiple column index); Oracle allows any number of rows to contain the value NULL for the complete unique key.

Oracle Microsoft SQL Server
CREATE TABLE DEPT_ADMIN.DEPT
(DEPT VARCHAR2(4) NOT NULL,
DNAME VARCHAR2(30) NOT NULL,
CONSTRAINT DEPT_DEPT_PK
     PRIMARY KEY (DEPT)
     USING INDEX TABLESPACE
USER_DATA
     PCTFREE 0 STORAGE (
     INITIAL 10K NEXT 10K
     MINEXTENTS 1 MAXEXTENTS
UNLIMITED),
CONSTRAINT DEPT_DNAME_UNIQUE)
     UNIQUE (DNAME)
     USING INDEX TABLESPACE
USER_DATA
     PCTFREE 0 STORAGE (
     INITIAL 10K NEXT 10K
     MINEXTENTS 1 MAXEXTENTS
UNLIMITED)
)

CREATE TABLE
USER_DB.DEPT_ADMIN.DEPT
(DEPT VARCHAR(4) NOT NULL,
DNAME     VARCHAR(30) NOT
NULL,
CONSTRAINT DEPT_DEPT_PK
     PRIMARY KEY CLUSTERED
(DEPT),
CONSTRAINT DEPT_DNAME_UNIQUE
     UNIQUE NONCLUSTERED
(DNAME)
)

Adding and Removing Constraints

Disabling constraints can improve database performance and streamline the data replication process. For example, when you rebuild or replicate table data at a remote site, you do not have to repeat constraint checks because the integrity of the data was checked when it was originally entered into the table. You can program an Oracle application to disable and enable constraints (except for PRIMARY KEYand UNIQUE). It is recommended that you use the NOT FOR REPLICATION clause to suspend column-level, foreign key, and CHECK constraints during replication.

If you are not replicating data and need to remove a constraint, you can do so in Microsoft SQL Server using the CHECK and WITH NOCHECK options with the ALTER TABLE statement. With SQL Server, you can defer all the table constraints by using the ALL keyword with the NOCHECK clause. If your Oracle application uses the CASCADE option to disable or drop PRIMARY KEY or UNIQUE constraints, you may need to rewrite some code because the CASCADE option disables or drops both the parent and any related child integrity constraints. The SQL Server-based application must be modified to first drop the child constraints and then the parent constraints. The ALTER TABLE syntax that adds and drops constraints is almost identical for Oracle and SQL Server.

Generating Unique Values

In an SQL Server table, one column can be defined as an identity column, which is an auto-incrementing integer field. SQL Server automatically tracks inserts and adds the value for the identity column to the record. If your application uses Oracle SEQUENCE to generate unique integers for a column, then you can replace it with the identity field (Table 26.5).

Table 26.5. SQL Server's IDENTITY Property
Category | Microsoft SQL Server IDENTITY
Syntax
CREATE TABLE new_employees
(Empid int IDENTITY (1,1), Employee_Name
varchar(60),
CONSTRAINT Emp_PK PRIMARY KEY
(Empid)
)
If increment interval is 5:
CREATE TABLE new_employees
(Empid int IDENTITY (1,5), Employee_Name
varchar(60),
CONSTRAINT Emp_PK PRIMARY KEY
(Empid)
)
Identity columns per table | One
Null values allowed | No
Use of default constraints, values | Cannot be used
Enforcing uniqueness | Yes
Querying for maximum current identity number after an INSERT, SELECT INTO, or bulk copy statement completes | @@IDENTITY (function)
Returns the seed value specified during the creation of an identity column | IDENT_SEED('table_name')
Returns the increment value specified during the creation of an identity column | IDENT_INCR('table_name')
SELECT syntax | The keyword IDENTITYCOL can be used in place of a column name when you reference a column that has the IDENTITY property, in SELECT, INSERT, UPDATE, and DELETE statements.

Although the IDENTITY property automates row numbering within one table, separate tables, each with its own identifier column, can generate the same values. This is because the IDENTITY property is guaranteed to be unique only for the table on which it is used. If an application must generate an identifier column that is unique across the entire database or to every database on every networked computer in the world, use the ROWGUIDCOL property, the uniqueidentifier data type, and the NEWID function. SQL Server uses globally unique identifier columns for merge replication to ensure that rows are uniquely identified across multiple copies of the table. If your application uses an Oracle SEQUENCE to generate a unique value that is then concatenated with another value to produce a unique string, you will have to create some custom code, in either a trigger or a stored procedure, that will generate the concatenated string for you.

Domain Integrety

Domain integrity enforces valid entries for a given column. Domain integrity is enforced by restricting the type (through data types), the format (through CHECK constraints), or the range of possible values (through REFERENCE and CHECK constraints).

DEFAULT and CHECK constraints

Oracle treats a default as a column property, and Microsoft SQL Server treats a default as a constraint. The SQL Server DEFAULT constraint can contain constant values, built-in functions that do not take arguments (niladic functions), or NULL. To easily move the Oracle DEFAULT column property, you should define DEFAULT constraints at the column level in SQL Server without applying constraint names. SQL Server generates a unique name for each DEFAULT constraint. The syntax used to define CHECK constraints is the same in Oracle and SQL Server. The search condition must evaluate to a Boolean expression and cannot contain subqueries. A column-level CHECK constraint can reference only the constrained column, and a table-level check constraint can reference only columns of the constrained table. Multiple CHECK constraints can be defined for a table. SQL Server syntax allows only one column-level CHECK constraint to be created on a column in a CREATE TABLE statement, and the constraint can have multiple conditions.

Nullability

Microsoft SQL Server and Oracle create column constraints to enforce nullability. An Oracle column defaults to NULL, unless NOT NULL is specified in the CREATE TABLE or ALTER TABLE statements. In Microsoft SQL Server, database and session settings can override the nullability of the data type used in a column definition.

All SQL scripts (whether Oracle or SQL Server) should explicitly define both NULL and NOT NULL for each column. When not explicitly specified, column nullability follows these rules.

  • Column is defined with a user-defined data type. SQL Server uses the nullability specified when the data type was created. Use the sp_help system stored procedure to get the data type's default nullability.

  • Column is defined with a system-supplied data type. If the system-supplied data type has only one option, it takes precedence. Currently, the bit data type can be defined only as NOT NULL. If any session settings are ON (set with the SET), then if ANSI_NULL_DFLT_ON is ON, NULL is assigned; if ANSI_NULL_DFLT_OFF is ON, NOT NULL is assigned. If any database settings are configured (changed with the ALTER DATABASE statement), then if ANSI NULL DEFAULT is TRUE, NULL is assigned; if ANSI NULL DEFAULT is FALSE, NOT NULL is assigned.

  • NULL/NOT NULL (not defined). When not explicitly defined (neither of the ANSI_NULL_DFLT options are set), the session has not been changed, and the database is set to the default (ANSI NULL DEFAULT is FALSE), then SQL Server assigns it NOT NULL.

Referential Integrity

The rules for defining foreign keys are similar in each RDBMS. The number of columns and data type of each column specified in the foreign key clause must match the REFERENCES clause. A non-null value entered in this column(s) must exist in the table and column(s) defined in the REFERENCES clause, and the referenced table's columns must have a PRIMARY KEY or UNIQUE constraint. Microsoft SQL Server constraints provide the ability to reference tables within the same database. To implement referential integrity across databases, use table-based triggers. Both Oracle and SQL Server support self-referenced tables, tables in which a reference (foreign key) can be placed against one or more columns on the same table. For example, the column prerequisite in the CLASS table can reference the column ccode in the CLASS table to ensure that a valid course code is entered as a course prerequisite.

In SQL Server 2000, foreign keys have an ON DELETE clause that is used to define what action should be taken if a candidate key to which the foreign key is pointing is deleted. The NO ACTION option causes the delete to fail with an error. The CASCADE option cascades the delete to any rows that reference the data within the FOREIGN KEY constraint.

User-Defined Integrity

User-defined integrity allows you to define specific business rules that do not fall into one of the other integrity categories. Microsoft SQL Server stored procedures use the CREATE PROCEDURE statement to accept and return user-supplied parameters. With the exception of temporary stored procedures, stored procedures are created in the current database. Following are the syntaxes for Oracle and SQL Server.

Oracle Microsoft SQL Server
CREATE OR REPLACE PROCEDURE
[user.]procedure
    [(argument [IN | OUT] datatype
    [, argument [IN | OUT] datatype]
{IS | AS}  block

CREATE PROC[EDURE]
procedure_name [;number]
    [
        {@parameter data_type}
[VARYING] [= default] [OUTPUT]
    ]
    [,...n]
[WITH
    {RECOMPILE   | ENCRYPTION  |
RECOMPILE, ENCRYPTION}  ]
[FOR REPLICATION]
AS
    sql_statement [...n]

In SQL Server, temporary procedures are created in the tempdb database by prefacing procedure_name with a single number sign (#procedure_name) for local temporary procedures, a double number sign (##procedure_name) for global temporary procedures. A local temporary procedure can be used only by the user who created it; permission to execute a local temporary procedure cannot be granted to other users. Local temporary procedures are automatically dropped at the end of the user session.

A global temporary procedure is available to all SQL Server users. If a global temporary procedure is created, all users can access it, and permissions cannot be explicitly revoked. Global temporary procedures are dropped at the end of the last user session using the procedure.

SQL Server stored procedures can be nested up to 32 levels. The nesting level is incremented when the called procedure starts execution, and it is decremented when the called procedure finishes execution. SQL Server does not support constructs similar to Oracle packages, and it does not support the CREATE OR REPLACE option for creating stored procedures. Instead, SQL Server supports either the CREATE or ALTER statements to create or modify stored procedures.

Delaying the Execution of a Stored Procedure

Microsoft SQL Server provides WAITFOR, which allows developers to specify a time, time interval, or event that triggers the execution of a statement block, stored procedure, or transaction. This is the Transact-SQL equivalent to the Oracle dbms_lock.sleep:

WAITFOR {DELAY 'time' | TIME 'time'}

where

DELAY instructs Microsoft SQL Server to wait until the specified amount of time has passed, up to a maximum of 24 hours.

'time' is the amount of time to wait. time can be specified in one of the acceptable formats for datetime data, or it can be specified as a local variable. Dates cannot be specified; therefore, the data portion of the datetime value is not allowed.

TIME instructs SQL Server to wait until the specified time.

For example,

BEGIN
 WAITFOR TIME '22:20'
 EXECUTE USP_Update_ALL
END

Specifying Parameters in a Stored Procedure

To specify a parameter within a stored procedure, use the following syntaxes.

Oracle Microsoft SQL Server
Varname datatype
DEFAULT <value>;

{@parameter data_type}  [VARYING]
[= default] [OUTPUT]

Triggers

Both Oracle and Microsoft SQL Server have triggers, which have some differences in their implementations (Table 26.6).

Triggers can be created to execute either after (AFTER trigger) the INSERT, UPDATE, or DELETE statement or INSTEAD OF the statement. If you need the functionality of a BEFORE trigger from Oracle, you have to add the INSERT, UPDATE, or DELETE statement to the logic within the INSTEAD OF trigger.

Table 26.6. Triggers
Description | Oracle | Microsoft SQL Server
Number of triggers per table | Unlimited | Unlimited
Triggers executed before INSERT, UPDATE, DELETE | Yes | Yes. This functionality can be created with the INSTEAD OF option.
Triggers executed after INSERT, UPDATE, DELETE | Yes | Yes
Statement-level triggers | Yes | Yes
Row-level triggers | Yes | No
Constraints checked prior to execution | Yes, unless trigger is disabled. | Yes. In addition, this is an option in Data Transformation Services.
Referring to old or previous values in an UPDATE or DELETE trigger | :old | DELETED.column
Referring to new values in an INSERT trigger | :new | INSERTED.column
Disabling triggers | ALTER TRIGGER | Option in Data Transformation Services

The deleted and inserted tables are logical (conceptual) tables created by SQL Server for trigger statements. They are structurally similar to the table on which the trigger is defined, and they hold the old values or new values of the rows that might be changed by the user action. The tables track row-level changes in Transact-SQL. These tables provide the same functionality as Oracle row-level triggers. When an INSERT, UPDATE, or DELETE statement is executed in SQL Server, rows are added to the trigger table and to the inserted and deleted table(s) simultaneously.

The inserted and deleted tables are identical to the trigger table. They have the same column names and the same data types. The inserted and deleted tables can be examined by the trigger to determine what types of trigger actions should be carried out. The inserted table is used with the INSERT and UPDATE statements. The deleted table is used with DELETE and UPDATE statements.

The UPDATE statement uses both the inserted and deleted tables because SQL Server always deletes the old row and inserts a new row whenever an UPDATE operation is performed. Consequently, when an UPDATE is performed, the rows in the inserted table are always duplicates of the rows in the deleted table. You can create a trigger only in the current database, though you can reference objects outside the current database. If you use an owner name to qualify a trigger, qualify the table name the same way.

Multiple AFTER triggers can be defined for each data modification event for a table. However, there can be only one INSTEAD OF trigger defined for a table.

Triggers can be nested 32 levels deep. If a trigger changes a table on which there is another trigger, the second trigger is activated and can then call a third trigger, and so on. If any trigger in the chain sets off an infinite loop, the nesting level is exceeded and the trigger is canceled. Additionally, if an update trigger on one column of a table results in an update to another column, the update trigger is activated only once. SQL Server declarative referential integrity (DRI) does not provide cross-database referential integrity. If cross-database referential integrity is required, use triggers. The following statements are not allowed in a Transact-SQL trigger:

  • CREATE statements (DATABASE, TABLE, INDEX, PROCEDURE, DEFAULT, RULE, TRIGGER, SCHEMA, and VIEW)

  • DROP statements (TRIGGER, INDEX, TABLE, PROCEDURE, DATABASE, VIEW, DEFAULT, RULE)

  • ALTER statements (DATABASE, TABLE, VIEW, PROCEDURE, TRIGGER)

  • TRUNCATE TABLE

  • GRANT, REVOKE, DENY

  • UPDATE STATISTICS

  • RECONFIGURE

  • UPDATE STATISTICS

  • RESTORE DATABASE, RESTORE LOG

  • LOAD LOG, DATABASE

  • DISK statements

  • SELECT INTO (because it creates a table)


Transactions, Locking, and Concurrency

Probably the most controversial issue when migrating from Oracle Applications to SQL Server is transactions, locking, and concurrency since it's so different in the two RDBMS. This section presents the differences between the locking processes and concurrency issues in both database types.

Transactions

In Oracle, a transaction is started automatically when an INSERT, UPDATE, or DELETE operation is performed. An application must issue a COMMIT command to save changes to the database. If a COMMIT is not performed, all changes are rolled back or undone automatically.

By default, SQL Server automatically performs a COMMIT statement after every insert, update, or delete operation. Because the data is automatically saved, you are unable to roll back any changes. You can start transactions in SQL Server as explicit, autocommit, or implicit transactions. Autocommit is the default behavior; you can use implicit or explicit transaction modes to change this default behavior.

Autocommit transactions are the default mode for SQL Server. Each Transact-SQL statement is committed when it completes. You do not have to specify any statements to control transactions.

As in Oracle, an implicit transaction is started whenever an INSERT, UPDATE, DELETE, or other data manipulating function is performed. To allow implicit transactions, use the SET IMPLICIT_TRANSACTIONS ON statement. If this option is ON and there are no outstanding transactions, every SQL statement automatically starts a transaction. If there is an open transaction, no new transaction is started. The open transaction must be committed by the user explicitly with the COMMIT TRANSACTION statement for the changes to take effect and for all locks to be released.

An explicit transaction is a grouping of SQL statements surrounded by the following transaction delimiters. Note that BEGIN TRANSACTION and COMMIT TRANSACTION are required:

BEGIN TRANSACTION [transaction_name]
COMMIT TRANSACTION [transaction_name]
ROLLBACK TRANSACTION [transaction_name | savepoint_name]
SAVE TRANSACTION {savepoint_name | @savepoint_variable}

The SAVE TRANSACTION statement functions in the same way as the Oracle SAVEPOINT command, setting a savepoint in the transaction that allows partial rollbacks.

Transactions can be nested one within another. If this occurs, the outermost pair creates and commits the transaction, and the inner pairs track the nesting level. When a nested transaction is encountered, the @@TRANCOUNT function is incremented. Usually, this apparent transaction nesting occurs as stored procedures or triggers with BEGIN…COMMIT pairs calling each other. Although transactions can be nested, they have little effect on the behavior of ROLLBACK TRANSACTION statements.

In stored procedures and triggers, the number of BEGIN TRANSACTION statements must match the number of COMMIT TRANSACTION statements. A stored procedure or trigger that contains unpaired BEGIN TRANSACTION and COMMIT TRANSACTION statements produces an error message when executed. The syntax allows stored procedures and triggers to be called from within transactions if they contain BEGIN TRANSACTION and COMMIT TRANSACTION statements.

Wherever possible, break large transactions into smaller transactions. Make sure each transaction is well defined within a single batch. To minimize possible concurrency conflicts, transactions should not span multiple batches nor wait for user input. Grouping many Transact-SQL statements into one long-running transaction can negatively affect recovery time and cause concurrency problems.

When programming with ODBC, you can select either the implicit or the explicit transaction mode by using the SQLSetConnectOption function. An ODBC program's selection of one or the other depends on the AUTOCOMMIT connect option. If AUTOCOMMIT is ON (the default), you are in explicit mode. If AUTOCOMMIT is OFF, you are in implicit mode.

If you are issuing a script through SQL Query Analyzer or other query tools, you can either include the explicit BEGIN TRANSACTION statement shown previously or start the script with the SET IMPLICIT_TRANSACTIONS ON statement. The BEGIN TRANSACTION approach is more flexible, and the implicit approach is more compatible with Oracle.

Locking and Transaction Isolation

One of the key functions of a database management system is to ensure that multiple users can read and write records in the database without reading inconsistent sets of records due to in-progress changes and without overwriting each other's changes inadvertently. Oracle and SQL Server approach this task with different locking and isolation strategies. You must consider these differences when you convert an application from Oracle to SQL Server or the resulting application may scale poorly to high numbers of users.

Oracle uses a multiversion consistency model for all SQL statements that read data either explicitly or implicitly. In this model, data readers by default neither acquire locks nor wait for other locks to be released before reading rows of data. When a reader requests data that has been changed but not yet committed by other writers, Oracle recreates the old data by using its rollback segments to reconstruct a snapshot of rows.

Data writers in Oracle request locks on data that is updated, deleted, or inserted. These locks are held until the end of a transaction, and they prevent other users from overwriting uncommitted changes.

SQL Server, in contrast, uses shared locks to ensure that data readers only see committed data. These readers take and release shared locks as they read data. These shared locks do not affect other readers. A reader waits for a writer to commit the changes before reading a record. A reader holding shared locks also blocks a writer trying to update the same data.

Releasing locks quickly for applications that support high numbers of users is more important in SQL Server than in Oracle. Releasing locks quickly is usually a matter of keeping transactions short. If possible, a transaction should neither span multiple round-trips to the server nor wait for the user to respond. You also need to code your application to fetch data as quickly as possible because unfetched data scans can hold share locks at the server and thus block updaters.

Dynamic Locking

SQL Server uses a dynamic locking strategy to determine the most cost-effective locks. SQL Server automatically determines which locks are most appropriate when the query is executed, based on the characteristics of the schema and query. For example, to reduce the overhead of locking, the optimizer may choose page-level locks in an index when performing an index scan. Dynamic locking has the following advantages:

  • Simplified database administration: Database administrators no longer have to be concerned with adjusting lock escalation thresholds.

  • Increased performance: SQL Server minimizes system overhead by using locks appropriate to the task.

  • Increased efficiency: Application developers can concentrate on development, because SQL Server automatically adjusts locking.

Oracle's inability to escalate row-level locks can cause problems in queries that include the FOR UPDATE clause and in UPDATE statements that request many rows. For example, assume that the MyTable table has 100,000 rows, and an Oracle user issues the following statement (note that 100,000 rows are affected):

UPDATE MyTable set (col) = (value);

The Oracle RDBMS locks every row in the MyTable table, one row at a time; this can take quite a while, and can require many system resources. Oracle does not escalate the request to lock the entire table. The same statement in SQL Server will cause the (default) row-level locks to escalate to a table-level lock, which is both efficient and fast.

Changing Default Locking Behavior

Both Microsoft SQL Sever and Oracle use the same default transaction isolation level: READ COMMITTED. Both databases also allow the developer to request nondefault locking and isolation behavior. In Oracle, the most common mechanisms for this are the FOR UPDATE clause on a SELECT command, the SET TRANSACTION READ ONLY command, and the explicit LOCK TABLE command.

Because their locking and isolation strategies are so different, it is difficult to map these locking options directly between Oracle and SQL Server. To obtain a better understanding of this process, it is important to understand the options that SQL Server provides for changing its default locking behavior.

In SQL Server, the most common mechanisms for changing default locking behavior are the SET TRANSACTION ISOLATION LEVEL statement and the locking hints that are supported in the SELECT and UPDATE statements. The SET TRANSACTION ISOLATION LEVEL statement sets transaction isolation levels for the duration of a user's session. This becomes the default behavior for the session unless a locking hint is specified at the table level in the FROM clause of an SQL statement. The transaction isolation is set like this:

SET TRANSACTION ISOLATION LEVEL {
        READ COMMITTED
        | READ UNCOMMITTED
        | REPEATABLE READ
        | SERIALIZABLE
    }

The READ COMMITTED option is the SQL Server default. Shared locks are held while the data is being read to avoid dirty reads, but the data can be changed before the end of the transaction, resulting in nonrepeatable reads or phantom data.

READ UNCOMMITTED implements dirty read, or isolation level 0 locking, which means that no shared locks are issued and no exclusive locks are honored. When this option is set, it is possible to read uncommitted or dirty data; values in the data can be changed and rows can appear or disappear in the data set before the end of the transaction. This option has the same effect as setting NOLOCK on all tables in all SELECT statements in a transaction. This is the least restrictive of the four isolation levels.

In REPEATABLE READ, locks are placed on all data that is used in a query, preventing other users from updating the data, but new phantom rows can be inserted into the data set by another user and are included in later reads in the current transaction. Because concurrency is lower than the default isolation level, use this option only when necessary.

In SERIALIZABLE, a range lock is placed on the data set preventing other users from updating or inserting rows into the data set until the transaction is complete. This is the most restrictive of the four isolation levels. Because concurrency is lower, use this option only when necessary. This option has the same effect as setting HOLDLOCK on all tables in all SELECT statements in a transaction.

SQL Server implements all four SQL-92 standard transaction isolation levels; Oracle implements only READ COMMITTED (the default) and SERIALIZABLE.SQL Server does not directly support the non-SQL-92 standard READ ONLY transaction isolation level offered by Oracle. If a transaction in an application requires repeatable read behavior, you may need to use the SERIALIZABLE isolation level offered by SQL Server. If all of the database access is read only, you can improve performance by setting the SQL Server database option to READ ONLY.

The SELECT…FOR UPDATE statement in Oracle is used when an application needs to issue a positioned update or delete on a cursor using the WHERE CURRENT OF syntax. In this case, optionally remove the FOR UPDATE clause; SQL Server cursors are updatable by default.

SQL Server cursors usually do not hold locks under the fetched row. Rather, they use an optimistic concurrency strategy to prevent updates from overwriting each other. If one user attempts to update or delete a row that has been changed since it was read into the cursor, SQL Server detects the problem and issues an error message. The application can trap this error message and retry the update or delete as appropriate.

The optimistic technique supports higher concurrency in the normal case where conflicts between updaters are rare. If your application really needs to ensure that a row cannot be changed after it is fetched, you can use the UPDLOCK hint in your SELECT statement to do so. This hint does not block other readers, but it prevents any other potential writers from also obtaining an update lock on the data. When using ODBC, you can also achieve a similar effect using

SQLSETSTMTOPTION (...,SQL_CONCURRENCY)= SQL_CONCUR_LOCK

Either of these options reduces concurrency.

Handling Deadlocks

A deadlock occurs when one process locks a resource needed by another process, and the second process locks a page the first process needs. SQL Server automatically detects and resolves deadlocks. If a deadlock is found, the server terminates the user process that has completed the deadly embrace.

After every data modification, your program code should check for message number 1205, which indicates a deadlock. If this message number is returned, a deadlock has occurred and the transaction was rolled back. In this situation, your application must restart the transaction. Deadlocks can usually be avoided by using a few simple techniques:

  • Access tables in the same order in all parts of your application.

  • Use a clustered index on every table to force an explicit row ordering.

  • Keep transactions short.

Insert Statements: The Forgotten Culprit

The behavior of the insert statements is very different in Oracle and SQL Server. Suppose you have two tables, TableA and TableB, and there is a foreign key relationship between TableB and TableA. Now say that there are two different sessions started by two different users. In Session1, User1 starts a transaction and updates col4 in TableA to, say, a value of 20, where column1 (the primary key of TableA) = 2, and at the same time in Session2, User2 tries to insert a record in the TableB as shown in the example script that follows (he is trying to insert a record in TableB where the foreign key value is 2). SQL Server internally fires a select statement to validate whether the insert should be allowed in TableB or not and since writers block readers in SQL Server until User1 either commits/rollbacks the transaction in Session1, the transaction in Session2 for User2 is blocked until either a lock_timeout occurs; or if no lock timeout value is specified, the transaction just hangs and waits. Many people think that an insert statement does not hold locks. This is not true at all. Another step in this scenario is that if the insert fails, and the application does not have an error handler to trap error 1222 (for Lock timeout), then until the transaction in Session2 is committed or rolled back, the locks on TableB will not be released. So it is essential to have a rollback and an error handler in such cases where the insert is going to occur.

A workaround for this problem is to make the primary key in TableA nonclustered instead of a clustered index so that the index page is the only one that is locked. Under such a scenario, then, this problem won't occur. But that could mean a performance hit for the application, since most frequently queried columns and the join conditions are based on the primary key columns, and having them as a clustered index definitely helps to improve performance since the data is stored along with the table itself.

Example script
Create table TableA (Col1 int, Col2 varchar(10), constraint PK_TableA Primary Key
(Col1))
Go
Create table TableB (Col1 int, Col2 int, Col3 varchar(10), constraint PK_TableB
Primary Key (Col1, Col2))
Go
Alter table TableB Add constraint FK_TableB_TO_TableA foreign key (col1) references
TableA (Col1)
Go
Insert into TableA (col1, col2) values (1, 'Test')
Insert into TableA (col1, col2) values (2, 'Test2')
Insert into TableA (col1, col2) values (3, 'Test3')
Insert into TableA (col1, col2) values (4, 'Test4')
Insert into TableB (col1, col2, col3) values (1, 2, 'Btest')
Session1:
Begin Tran
Update TableA Set col2 = 'Test-A' where col1 = 2
-- Do not commit yet
Session 2:
Set Lock_TimeOut 5000
-- Setting the lock timeout to 5 seconds
Begin Tran
Insert into TableB (col1, col2, col3) values (2, 3, 'TestB')
-- this will hang till 5 seconds because of the lock time out setting and will
receive an error.
-- And till the transaction is committed or rolled back, it will acquire locks on
TableB (you can see —that using sp_lock).


Remote Transactions

To perform remote transactions in Oracle, you must have access to a remote database node with a database link. In SQL Server, you must have access to a remote server. A remote server is a server running SQL Server on the network that users can access by using their local server. When a server is set up as a remote server, users can use the system procedures and the stored procedures on it without explicitly logging in to it.

Remote servers are set up in pairs. You must configure both servers to recognize each other as remote servers. The name of each server must be added to its partner with the sp_addlinkedserver system stored procedure or the SQL Server Enterprise Manager.

After you set up a remote server, use the sp_addremotelogin system stored procedure or the SQL Server Enterprise Manager to set up remote login IDs for the users who must access that remote server. After this step is completed, you must grant permission to execute the stored procedures.

The EXECUTE statement is then used to run procedures on the remote server. This example executes the USP_Return_ALL stored procedure on the remote server SVR1 and stores the return status indicating success or failure in @retvalue1:

DECLARE @retvalue1 int
EXECUTE @retvalue = SVR1.Other_db.admin_account.USP_Return_All '111111111'


Distributed Transactions

Oracle automatically initiates a distributed transaction if changes are made to tables in two or more networked database nodes. SQL Server distributed transactions use the two-phase commit services of the Microsoft Distributed Transaction Coordinator (MS DTC) included with SQL Server.

By default, SQL Server must be instructed to participate in a distributed transaction. SQL Server participation in an MS DTC transaction can be started by either the BEGIN DISTRIBUTED TRANSACTION statement (it begins a new MS DTC transaction) or a client application calling MS DTC transaction interfaces directly. If the application cannot complete the transaction, the application program cancels it by using the ROLLBACK TRANSACTION statement. If the application fails or a participating resource manager fails, MS DTC cancels the transaction. MS DTC does not support distributed savepoints or the SAVE TRANSACTION statement. If an MS DTC transaction is terminated or rolled back, the entire transaction is rolled back to the beginning of the distributed transaction, regardless of any savepoints.


Two-Phase Commit Processing

The Oracle and MS DTC two-phase commit mechanisms are similar in operation. In the first phase of an SQL Server two-phase commit, the transaction manager requests each enlisted resource manager to prepare to commit. If any resource manager cannot do so, the transaction manager broadcasts an abort decision to everyone involved in the transaction.

If all resource managers can successfully prepare to commit, the transaction manager broadcasts the commit decision. This is the second phase of the commit process. While a resource manager is prepared, it is in doubt about whether the transaction is committed or terminated. MS DTC keeps a sequential log so that its commit or terminate decisions are durable. If a resource manager or transaction manager fails, they reconcile in-doubt transactions when they reconnect.


SQL Language Support

This section outlines the similarities and differences between Transact-SQL and PL/SQL language syntax and presents conversion strategies.

SELECT and Data Manipulation Statements

Use the following guidelines when moving Oracle DML statements and PL/SQL programs to SQL Server.

  1. Verify that the syntax of all SELECT, INSERT, UPDATE, and DELETE statements is valid. Make any required modifications.

  2. Change all outer joins to SQL-92 standard outer join syntax.

  3. Replace Oracle functions with the appropriate SQL Server functions.

  4. Check all comparison operators.

  5. Replace the "||" string concatenation operator with the "+" string concatenation operator.

  6. Replace PL/SQL programs with Transact-SQL programs.

  7. Change all PL/SQL cursors to either noncursor SELECT statements or Transact-SQL cursors.

  8. Replace PL/SQL procedures, functions, and packages with Transact-SQL procedures.

  9. Convert PL/SQL triggers to Transact-SQL triggers.

  10. Use the SET SHOWPLAN statement to tune your queries for performance.

SELECT Statements

The SELECT statement syntax used by Oracle and Microsoft SQL Server is similar except for the place where the optimization hints are placed.

Oracle Microsoft SQL Server
SELECT [/*+ optimizer_hints*/]
[ALL | DISTINCT] select_list
[FROM
{table_name | view_name | select_statement}]
[WHERE clause]
[GROUP BY group_by_expression]
[HAVING search_condition]
[START WITH ... CONNECT BY]
[{UNION | UNION ALL | INTERSECT |
   MINUS}  SELECT ...]
[ORDER BY clause]
[FOR UPDATE]

SELECT select_list
[INTO new_table_]
FROM table_source
[WHERE search_condition]
[ GROUP BY [ALL] group_by_expression
[,...n]
     [ WITH { CUBE | ROLLUP }  ]
[HAVING search_condition]
[ORDER BY order_expression [ASC | DESC]
]
In addition:
UNION operator
COMPUTE clause
FOR BROWSE clause
OPTION clause

Oracle-specific cost-based optimizer hints are not supported by SQL Server and must be removed. The recommended technique is to use SQL Server cost-based optimization.

SQL Server does not support the Oracle START WITH…CONNECT BY clause. You can replace this in SQL Server by creating a stored procedure that performs the same task. The Oracle INTERSECT and MINUS set operators are not supported by SQL Server. The SQL Server EXISTS and NOT EXISTS clauses can be used to accomplish the same result.

INSERT Statements

The INSERT statement syntaxes used by Oracle and Microsoft SQL Server are similar.

Oracle Microsoft SQL Server
INSERT INTO
{table_name | view_name | select_statement}
[(column_list)]
{values_list | select_statement}

INSERT [INTO]
{
  table_name [ [AS] table_alias] WITH (
<table_hint_limited> [...n])
     | view_name [ [AS] table_alias]
     | rowset_function_limited
    }

    {    [(column_list)]
        { VALUES ( {   DEFAULT
                        |     NULL
                        |     expression
                        } [,...n]
            )
        | derived_table
        | execute_statement
        }
    }
    | DEFAULT VALUES

The Transact-SQL language supports inserts into tables and views but does not support insert operations into SELECT statements. If your Oracle application code performs inserts into SELECT statements, it must be changed.

The Transact-SQL values_list parameter offers the SQL-92 standard keyword DEFAULT, which is not supported by Oracle. This keyword specifies that the default value for the column be used when an insert is performed. If a default value does not exist for the specified column, a NULL is inserted. If the column does not allow NULLs, an error message is returned. If the column is defined as a timestamp data type, the next sequential value is inserted.

The DEFAULT keyword cannot be used with an identity column. To generate the next sequential number, columns with the IDENTITY property must not be listed in the column_list or values_clause. You do not have to use the DEFAULT keyword to obtain the default value for a column. As in Oracle, if the column is not referenced in the column_list and it has a default value, the default value is placed in the column. This is the most compatible approach to use when performing the migration.

One useful Transact-SQL option (EXECute procedure_name) is to execute a procedure and pipe its output into a target table or view. Oracle does not allow this.

UPDATE Statements

Because Transact-SQL supports most of the syntax used by the Oracle UPDATE command, minimal revision is required.

Oracle Microsoft SQL Server
UPDATE
{table_name | view_name | select_statement}
SET [column_name(s) = {constant_value |
expression | select_statement | column_list |
variable_list]
{where_statement}

UPDATE
     {
     table_name [ [AS] table_alias] WITH (
 <table_hint_limited> [...n])
         | view_name [ [AS] table_alias]
         | rowset_function_limited
        }
        SET
        {column_name = {expression |
DEFAULT | NULL}
        | @variable = expression
        | @variable = column = expression }
[,...n]

    {{[FROM {<table_source>}  [, ...n] ]

        [WHERE
            <search_condition>] }
        |
        [WHERE CURRENT OF
        { { [GLOBAL] cursor_name }  |
cursor_variable_name}
        ] }
        [OPTION (<query_hint> [,...n] )]

The Transact-SQL UPDATE statement does not support update operations against SELECT statements. If your Oracle application code performs updates against SELECT statements, you can turn the SELECT statement into a view and then use the view name in the SQL Server UPDATE statement. The Oracle UPDATE command can use program variables only from within a PL/SQL block. The Transact-SQL language does not require the use of blocks to use variables. The keyword DEFAULT can be used to set a column to its default value in SQL Server. You cannot set a column to a default value with the Oracle UPDATE command.

Transact-SQL and Oracle SQL support the use of subqueries in an UPDATE statement. However, the Transact-SQL FROM clause can be used to create an UPDATE based on a join. This capability makes the UPDATE syntax more readable and in some cases can improve performance.

DELETE Statements

In most cases, DELETE statements do not need to be modified. If you perform deletes against SELECT statements in Oracle, you must modify the syntax for SQL Server, because Transact-SQL does not support this functionality.

Transact-SQL supports the use of subqueries in the WHERE clause, as well as joins in the FROM clause. The latter can produce more efficient statements.

Oracle Microsoft SQL Server
DELETE [FROM]
{table_name | view_name | select_statement}
[WHERE clause]

DELETE
    [FROM ]
        {
         table_name [ [AS] table_alias] WITH (
<table_hint_limited> [...n])
         | view_name [ [AS] table_alias]
         | rowset_function_limited
        }

        [ FROM {<table_source>}  [, ...n] ]
    [WHERE
        { <search_condition>
        |    { [ CURRENT OF
                {
                    { [ GLOBAL ] cursor_name }
                    | cursor_variable_name
                }
             ]
        }
    ]
    [OPTION (<query_hint> [,...n])]

TRUNCATE TABLE Statement

The TRUNCATE TABLE syntaxes used by Oracle and Microsoft SQL Server are similar. TRUNCATE TABLE is used to remove all the rows from a table. The table structure and all of its indexes continue to exist. Unlike the DELETE statement, which explicitly deletes each record of a table, TRUNCATE TABLE removes the data from the table by deallocating the data pages of the table and logging these deallocations in the transaction log. Because the rows are not explicitly deleted, DELETE triggers are not executed. If a table is referenced by a FOREIGN KEY constraint, it cannot be truncated.

Oracle Microsoft SQL Server
TRUNCATE TABLE table_name
[{DROP | REUSE}  STORAGE]
TRUNCATE TABLE table_name

In SQL Server, only the table owner can issue this statement. In Oracle, this command can be issued if you are the table owner or have the DELETE TABLE system privilege. The Oracle TRUNCATE TABLE command can optionally release the storage space occupied by the rows in the table. The SQL Server TRUNCATE TABLE statement always reclaims space occupied by the table data and its associated indexes.


Manipulating Data in Identity and Timestamp Columns

Oracle sequences are database objects that are not directly related to any given table or column. The relationship between a column and a sequence is implemented in the application by assigning the sequence value to a column programmatically. Therefore, Oracle does not enforce any rules when it works with sequences. However, in Microsoft SQL Server identity columns, values cannot be updated and the DEFAULT keyword cannot be used.

By default, data cannot be inserted directly into an identity column. The identity column automatically generates a unique, sequential number for each new row inserted in the table. This default can be overridden using the following SET statement:

SET IDENTITY_INSERT table_name ON

With IDENTITY_INSERT set to ON, the user can insert any value into the identity column of a new row. To prevent the entry of duplicate numbers, a unique index must be created against the column. The purpose of this statement is to allow a user to recreate a value for a row that has been deleted accidentally. The @@IDENTITY function can be used to obtain the last identity value.

The TRUNCATE TABLE statement resets an identity column to its original SEED value. If you do not want to reset the identity value for a column, use the DELETE statement without a WHERE clause instead of the TRUNCATE TABLE statement. You will have to evaluate how this affects your Oracle migration, because ORACLE SEQUENCES are not reset following the TRUNCATE TABLE command.

You can perform only inserts or deletes when working with timestamp columns. If you attempt to update a timestamp column, you receive this error message:

Msg 272, Level 16, State 1 Can't update a TIMESTAMP column.


Locking Requested Rows

Oracle uses the FOR UPDATE clause to lock rows specified in the SELECT command. You do not need to use the equivalent clause in Microsoft SQL Server because this is the default behavior.


Row Aggregates and the Compute Clause

The SQL Server COMPUTE clause is used to generate row aggregate functions (SUM, AVG, MIN, MAX, and COUNT), which appear as additional rows in the query results. The COMPUTE clause allows you to see detail and summary rows in one set of results. You can calculate summary values for subgroups, and you can calculate more than one aggregate function for the same group.

The Oracle SELECT command syntax does not support the COMPUTE clause. Nevertheless, the SQL Server COMPUTE clause works just like the COMPUTE command found in the Oracle SQL [image] Plus query tool.

Join Clauses

Microsoft SQL Server 2000 allows up to 256 tables—both temporary and permanent—to be joined in a join clause. There is no join limit in Oracle.

When using outer joins in Oracle, the outer join operator (+) is typically placed next to the child (foreign key) column in the join. The (+) identifies the column with fewer unique values. This always occurs unless the foreign key allows null values, in which case (+) can be placed on the parent (PRIMARY KEY or UNIQUE constraint) column. You cannot place the (+) on both sides of the equal sign (=).

With SQL Server, you can use the [image] = and = [image] outer join operators. The [image] is used to identify the column that has more unique values. If the child (foreign key) column does not allow null values, the [image] is placed on the parent (PRIMARY KEY or UNIQUE constraint) column side of the equal sign. The placement of the [image] is essentially reversed in Oracle. You cannot place the [image] on both sides of the equal sign (=).The [image] = and = [image] are considered legacy join operators. SQL Server also supports the SQL-92 standard join operators listed as follows. It is recommended that you use this syntax. The SQL-92 standard syntax is more powerful and has fewer restrictions than the [image] operators.

CROSS JOIN Cross product of two tables. Returns the same rows as if no WHERE clause was specified in an old-style join. This type of join is called a Cartesian-join in Oracle.

INNER Specifies that all inner rows be returned. Any unmatched rows are discarded. This is identical to a standard Oracle table join.

LEFT [OUTER] Specifies that all the left table outer rows be returned, even if no column matches are found. Operates just like an Oracle outer join (+).

RIGHT [OUTER] Specifies that all of the right table outer rows be returned, even if no column matches are found. Operates just like an Oracle outer join (+).

FULL [OUTER] If a row from either table does not match the selection criteria, this join specifies the row be included in the result set and its output columns that correspond to the other table be set to NULL. This would be the same as placing the Oracle outer join operator on both sides of the = (col1(+) = col2(+)), which is not allowed.


Reading and Modifying BLOBs

Microsoft SQL Server implements binary large objects (BLOBs) with text and image columns. Oracle implements BLOBs with LONG and LONG RAW columns. In Oracle, a SELECT command can query the values in LONG and LONG RAW columns.

In SQL Server, you can use a standard Transact-SQL statement or the specialized READTEXT statement to read data in text and image columns. The READTEXT statement allows you to read partial sections of a text or image column. Oracle does not provide an equivalent statement for working with LONG and LONG RAW columns.

The READTEXT statement makes use of a text_pointer, which can be obtained using the TEXTPTR function. The TEXTPTR function returns a pointer to the text or image column in the specified row or to the text or image column in the last row returned by the query if more than one row is returned. Because the TEXTPTR function returns a 16-byte binary string, it is best to declare a local variable to hold the text pointer and then use the variable with READTEXT.

The READTEXT statement specifies how many bytes to return. The value in the @@TEXTSIZE function, which is the limit on the number of characters or bytes to be returned, supersedes the size specified by the READTEXT statement if it is less than the specified size for READTEXT.

The SET statement can be used with the TEXTSIZE parameter to specify the size, in bytes, of text data to be returned with a SELECT statement. If you specify a TEXTSIZE of 0, the size is reset to the default (4 KB). Setting the TEXTSIZE parameter affects the @@TEXTSIZE function. The SQL Server ODBC driver automatically sets the TEXTSIZE parameter when the SQL_MAX_LENGTH statement option is changed.

In Oracle, UPDATE and INSERT commands are used to change values in LONG and LONG RAW columns. In SQL Server, you can use standard UPDATE and INSERT statements, or you can use the UPDATETEXT and WRITETEXT statements. Both UPDATETEXT and WRITETEXT allow a nonlogged option, and UPDATETEXT allows for partial updating of a text or image column.The UPDATETEXT statement can be used to replace existing data, delete existing data, or insert new data. Newly inserted data can be a constant value, table name, column name, or text pointer. The WRITETEXT statement completely overwrites any existing data in the column it affects. Use WRITETEXT to replace text data and UPDATETEXT to modify text data. The UPDATETEXT statement is more flexible because it changes only a portion of a text of image value rather than the entire value.


Functions

Tables 26.7–26.12 show the relationship between Oracle and SQL Server scalar-valued and aggregate functions. Although the names appear to be the same, the functions vary in numbers and types of arguments. Also, functions that are supplied only by Microsoft SQL Server are not mentioned in this list because this chapter is limited to easing migration from existing Oracle applications. Examples of functions not supported by Oracle are degrees (DEGREES), PI (PI), and random number (RAND).

Table 26.7. Number/Mathematical functions
Function | Oracle | SQL Server
Absolute value | ABS | ABS
Arc cosine | ACOS | ACOS
Arc sine | ASIN | ASIN
Arc tangent of n | ATAN | ATAN
Arc tangent of n and m | ATAN2 | ATN2
Smallest integer >= value | CEIL | CEILING
Cosine | COS | COS
Hyperbolic cosine | COSH | COT
Exponential value | EXP | EXP
Largest integer <= value | FLOOR | FLOOR
Natural logarithm | LN | LOG
Logarithm, any base | LOG(N) | N/A
Logarithm, base 10 | LOG(10) | LOG10
Modulus (remainder) | MOD | USE MODULO (%) OPERATOR
Power | POWER | POWER
Random number | N/A | RAND
Round | ROUND | ROUND
Sign of number | SIGN | SIGN
Sine | SIN | SIN
Hyperbolic sine | SINH | N/A
Square root | SQRT | SQRT
Tangent | TAN | TAN
Hyperbolic tangent | TANH | N/A
Truncate | TRUNC | N/A
Largest number in list | GREATEST | N/A (See the code provided on CD-ROM for a Greatest function created using User-defined functions.)
Smallest number in list | LEAST | N/A
Convert number if NULL | NVL | ISNULL

Table 26.8. Character functions
Function | Oracle | SQL Server
Convert character to ASCII | ASCII | ASCII
String concatenate | CONCAT | (expression + expression)
Convert ASCII to character | CHR | CHAR
Return starting point of character in character string (from left) | INSTR | CHARINDEX
Convert characters to lowercase | LOWER | LOWER
Convert characters to uppercase | UPPER | UPPER
Pad left side of character string | LPAD | N/A
Remove leading blanks | LTRIM | LTRIM
Remove trailing blanks | RTRIM | RTRIM
Starting point of pattern in character string | INSTR | PATINDEX
Repeat character string multiple times | RPAD | REPLICATE
Phonetic representation of character string | SOUNDEX | SOUNDEX
String of repeated spaces | RPAD | SPACE
Character data converted from numeric data | TO_CHAR | STR
Substring | SUBSTR | SUBSTRING
Replace characters | REPLACE | STUFF
Capitalize first letter of each word in string | INITCAP | N/A
Translate character string | TRANSLATE | N/A
Length of character string | LENGTH | DATALENGTH or LEN
Greatest character string in list | GREATEST | N/A
Least character string in list | LEAST | N/A
Convert string if NULL | NVL | ISNULL

Table 26.9. Date functions
Function | Oracle | Microsoft SQL Server
Date addition | (date column +/– value) or ADD_MONTHS | DATEADD
Difference between dates | (date column +/– value) or MONTHS_BETWEEN | DATEDIFF
Current date and time | SYSDATE | GETDATE()
Last day of month | LAST_DAY | N/A
Time zone conversion | NEW_TIME | N/A
First weekday after date | NEXT_DAY | N/A
Character string representation of date | TO_CHAR | DATENAME
Integer representation of date | TO_NUMBER(TO_CHAR)) | DATEPART
Date round | ROUND | CONVERT
Date truncate | TRUNC | CONVERT
Character string to date | TO_DATE | CONVERT
Convert date if NULL | NVL | ISNULL

Table 26.10. Conversion functions
Function | Oracle | Microsoft SQL Server
Number to character | TO_CHAR | CONVERT
Character to number | TO_NUMBER | CONVERT
Date to character | TO_CHAR | CONVERT
Character to date | TO_DATE | CONVERT
Hex to binary | HEX_TO_RAW | CONVERT
Binary to hex | RAW_TO_HEX | CONVERT

Table 26.11. Other Row-Level functions
Function | Oracle | Microsoft SQL Server
Return first non-null expression | DECODE | COALESCE
Current sequence value | CURRVAL | N/A
Next sequence value | NEXTVAL | N/A
If exp1 = exp2, return null | DECODE | NULLIF
User's login ID number | UID | SUSER_ID
User's login name | USER | SUSER_NAME
User's database ID number | UID | USER_ID
User's database name | USER | USER_NAME
Current user | CURRENT_USER | CURRENT_USER
User environment (audit trail) | USERENV | N/A
Level in CONNECT BY clause | LEVEL | N/A

Table 26.12. Aggregate functions
Function | Oracle | Microsoft SQL Server
Average | AVG | AVG
Count | COUNT | COUNT
Maximum | MAX | MAX
Minimum | MIN | MIN
Standard deviation | STDDEV | STDEV or STDEVP
Summation | SUM | Same
Variance | VARIANCE | VAR or VARP


Conditional Tests

Both the Oracle DECODE statement and the Microsoft SQL Server CASE expression perform conditional tests. When the value in test_value matches any following expression, the related value is returned. If no match is found, the default_value is returned. If no default_value is specified, both DECODE and CASE return NULL if there is no match. Following is the syntax as well as an example of a converted DECODE command.

Oracle Microsoft SQL Server
DECODE (test_value,
  expression1, value1
  [[,expression2, value2] [...]]
  [,default_value]
)

CASE test_value
WHEN expression1 THEN value1
[[WHEN expression2 THEN value2] [...]]
[ELSE default_value]
     END

The CASE expression can support the use of SELECT statements for performing Boolean tests, something the DECODE command does not allow. Beginning with version 8.1.6, Oracle also supports the CASE statements, so if you are using that version and have coded in case statements, you don't need to change anything.


Converting Values to Different Data Types

The Microsoft SQL Server CONVERT and CAST functions are multiple-purpose conversion functions. They provide similar functionality, converting an expression of one data type to another data type and supporting a variety of special date formats:

CAST (expression AS data_type)
CONVERT (data type[(length)], expression [, style])

CAST is an SQL-92 standard function. These functions perform the same operations as the Oracle TO_CHAR, TO_NUMBER, TO_DATE, HEXTORAW, and RAWTOHEX functions. The data type is any system data type into which the expression is to be converted. User-defined data types cannot be used. The length parameter is optional and is used with char, varchar, binary, and varbinary data types. The maximum allowable length is 8000.

Notice how character strings are converted to dates. In Oracle, the default date format model is "DD-MON-YY." If you use any other format, you must provide an appropriate date format model. The CONVERT function automatically converts standard date formats without the need for a format model. When you convert from a date to a character string, the default output for the CONVERT function is "dd mon yyyy hh:mm:ss:mmm(24h)." A numeric style code is used to format the output to other types of date format models.

Table 26.13. Converting values to different data types
Conversion | Oracle | Microsoft SQL Server
Character to number | TO_NUMBER('10') | CONVERT(numeric, '10')
Number to character | TO_CHAR(10) | CONVERT(char, 10)
Character to date

TO_DATE('04-JUL-97')

TO_DATE('04-JUL-1997', 'dd-mon-yyyy')

TO_DATE('July 4, 1997', 'Month dd, yyyy')

CONVERT(datetime, '04-JUL-97')

CONVERT (datetime, '04-JUL-1997')

CONVERT (datetime, 'July 4, 1997')
Date to character

TO_CHAR(sysdate)

TO_CHAR(sysdate, 'dd mon yyyy')

TO_CHAR(sysdate, 'mm/dd/yyyy')

CONVERT(char, GETDATE())

CONVERT(char, GETDATE (), 106)

CONVERT(char, GETDATE (), 101)
Hex to binary | HEXTORAW('1F') | CONVERT(binary, '1F')
Binary to hex | RAWTOHEX(binary_column) | CONVERT(char, binary_column)


User-Defined Functions

The syntaxes to create a user-defined function that returns a scalar data type are similar in Oracle and SQL Server 2000. SQL Server user-defined functions also can return a table data type. These functions can be either in-line functions, consisting of a simple SELECT statement, or multistatement functions, consisting of many statements used to build a table. Both functions can accept parameters and can be accessed in the FROM clause of a Transact-SQL statement. These functions can provide a powerful alternative both to using stored procedures, because their result set can be accessed directly within a Transact-SQL statement, and to using views, because they can accept parameters to narrow down a result set.

Following is the syntax to create the user-defined function types that return tables.

Syntax for an inline-table function

CREATE FUNCTION [ owner_name. ] function_name
   ( [ { @parameter_name [AS] scalar_parameter_data_type [ = default ] }
[ ,...n ] ] )
RETURNS TABLE
[ WITH <function_option > [ [,] ...n ] ]
[ AS ]
RETURN [ ( ] select-stmt [ ) ]

Syntax for a multistatement table function

CREATE FUNCTION [ owner_name. ] function_name
    ( [ { @parameter_name [AS] scalar_parameter_data_type [ = default ] }
[ ,...n ] ] )
RETURNS @return_variable TABLE < table_type_definition >
 [ WITH < function_option > [ [,] ...n ] ]
 [ AS ]
BEGIN
    function_body
    RETURN
END


Comparison Operators

Oracle and Microsoft SQL Server comparison operators are nearly identical (Table 26.14).

Table 26.14. Operations symbols
Operator | Oracle | Microsoft SQL Server
Equal to = =
Greater than > >
Less than < <
Greater than or equal to >= >=
Less than or equal to <= <=
Not equal to !=, <>,^= !=, <>,^=
Not greater than, not less than N/A !> , !<
In any member in set | IN | IN
Not in any member in set | NOT IN | NOT IN
Any value in set | ANY, SOME | ANY, SOME
Referring to all values in set | != ALL, <> ALL, < ALL, > ALL, <= ALL, >= ALL | != ALL, <> ALL, < ALL, > ALL, <= ALL, >= ALL
Like pattern | LIKE | LIKE
Not like pattern | NOT LIKE | NOT LIKE
Value between x and y | BETWEEN x AND y | BETWEEN x AND y
Value not between | NOT BETWEEN | NOT BETWEEN
Value exists | EXISTS | EXISTS
Value does not exist | NOT EXISTS | NOT EXISTS
Value is / is not NULL | IS NULL, IS NOT NULL | IS NULL, IS NOT NULL. Also supports = NULL, != NULL for backward compatibility (not recommended).

Pattern Matches

The SQL Server LIKE keyword offers useful wildcard search options that are not supported by Oracle. In addition to supporting the % and _ wildcard characters common to both RDBMSs, the [ ] and [^] characters are also supported by SQL Server.

The [ ] character set is used to search for any single character within a specified range. For example, if you search for the characters a through f in a single character position, you can specify this with LIKE '[a-f]' or LIKE '[abcdef]'.

The [^] wildcard character set is used to specify characters not in the specified range. For example, if any character except a through f is acceptable, you use LIKE '[^a - f]' or LIKE '[^abcdef]'.

Using NULL in Comparisons

Although Microsoft SQL Server traditionally has supported the SQL-92 standard as well as some nonstandard NULL behaviors, it supports the use of NULL in Oracle. SET ANSI_NULLS should be set to ON for executing distributed queries. The SQL Server ODBC Driver and OLE DB Provider for SQL Server automatically SET ANSI_NULLS to ON when connecting. This setting can be configured in ODBC data sources, in ODBC connection attributes, or in OLE DB connection properties that are set in the application before connecting to SQL Server. SET ANSI_NULLS defaults to OFF for connections from DB-Library applications. When SET ANSI_DEFAULTS is ON, SET ANSI_NULLS is enabled.


String Concatenation

Oracle uses two pipe symbols (||) as the string concatenation operator, and SQL Server uses the plus sign (+). This difference requires minor revision in your application program code.

Oracle Microsoft SQL Server
SELECT FNAME||' '||LNAME AS NAME
FROM DATA.USER_MASTER

SELECT FNAME +' '+ LNAME AS
NAME
FROM DATA.USER_MASTER


Control-of-Flow Language

The control-of-flow language controls the flow of execution of SQL statements, statement blocks, and stored procedures. PL/SQL and Transact-SQL provide many of the same constructs, although there are some syntax differences.

The keywords supported by each RDBMS are shown in Table 26.15.

Table 26.15. Keywords
Statement | Oracle PL/SQL | Transact-SQL
Declare variables | DECLARE | DECLARE
Statement block | BEGIN…END; | BEGIN…END
Conditional processing

IF…THEN,

ELSIF…THEN,

ELSE

ENDIF;

IF…[BEGIN…END]

ELSE <condition>

[BEGIN…END]

ELSE IF <condition>

CASE expression
Unconditional exit | RETURN | RETURN
Unconditional exit to the statement following the end of the current program block | EXIT | BREAK
Restarts a WHILE loop | N/A | CONTINUE
Wait for a specified interval | N/A (dbms_lock.sleep) | WAITFOR
Loop control

WHILE LOOP…END LOOP;

LABEL…GOTO LABEL;

FOR…END LOOP;

LOOP…END LOOP;

WHILE <condition>

BEGIN… END

LABEL…GOTO

LABEL
Program comments | / [image] … /, — | / [image] … /, —
Print output | RDBMS_OUTPUT.PUT_LINE | PRINT
Raise program error | RAISE_APPLICATION_ERROR | RAISERROR
Execute program | EXECUTE | EXECUTE
Statement terminator | Semicolon (;) | N/A

Declaring Variables

Transact-SQL and PL/SQL variables are created with the DECLARE keyword. Transact-SQL variables are identified with @ and, like PL/SQL variables, are initialized to a null value when they are first created.

Oracle Microsoft SQL Server
DECLARE
SSN CHAR(9);
FNAME VARCHAR2(12);
LNAME VARCHAR2(20);
BIRTH_DATE DATE;

DECLARE
@SSN CHAR(9),
@FNAME VARCHAR2(12),
@LNAME VARCHAR2(20),
@BIRTH_DATE DA

Transact-SQL does not support the %TYPE and %ROWTYPE variable data type definitions. A Transact-SQL variable cannot be initialized in the DECLARE command. The Oracle NOT NULL and CONSTANT keywords cannot be used in Microsoft SQL Server data type definitions. Like Oracle LONG and LONG RAW data types, text and image data types cannot be used for variable declarations. Additionally, the PL/SQL style record and table definitions are not supported.

Assigning Variables

Oracle and Microsoft SQL Server offer the following ways to assign values to local variables.

Oracle Microsoft SQL Server
Assignment operator (:=)
SELECT...INTO syntax for selecting column
values from a single row

FETCH...INTO syntax

SET @local_variable = value
SELECT @local_varialbe = expression
[FROM...] for assigning a literal value,
an expression involving other local variables, or
a column value from a single row
FETCH...INTO syntax

Statement Blocks

Oracle PL/SQL and Microsoft SQL Server Transact-SQL support the use of BEGIN…END terminology to specify statement blocks. Transact-SQL does not require the use of a statement block following the DECLARE statement. The BEGIN…END statement blocks are required in Microsoft SQL Server for IF statements and WHILE loops if more than one statement is executed.

Oracle Microsoft SQL Server
DECLARE
     DECLARE VARIABLES ...
BEGIN — THIS IS REQUIRED SYNTAX
     PROGRAM_STATEMENTS ...
     IF ...THEN
          STATEMENT1;
          STATEMENT2;
          STATEMENTN;
END IF;
WHILE ... LOOP
          STATEMENT1;
          STATEMENT2;
          STATEMENTN;
END LOOP;
END; — THIS IS REQUIRED SYNTAX

DECLARE
     DECLARE VARIABLES ...
BEGIN — THIS IS OPTIONAL SYNTAX
     PROGRAM_STATEMENTS ...
     IF ...
     BEGIN
          STATEMENT1
          STATEMENT2
          STATEMENTN
          END
WHILE ...
          BEGIN
          STATEMENT1
          STATEMENT2
          STATEMENTN
     END
END — THIS IS REQUIRED SYNTAX

Conditional Processing

The Microsoft SQL Server Transact-SQL conditional statement includes IF and ELSE rather than the ELSIF statement in Oracle PL/SQL. Multiple IF statements can be nested to achieve the same effect. For extensive conditional tests, the CASE expression may be easier to read.

Repeated Statement Execution (Looping)

Oracle PL/SQL provides the unconditional LOOP and FOR LOOP. Transact-SQL offers the WHILE loop and the GOTO statement for looping purposes.

WHILE Boolean_expression
     {sql_statement | statement_block}
     [BREAK] [CONTINUE]

The WHILE loop tests a Boolean expression for the repeated execution of one or more statements. The statement(s) are executed repeatedly as long as the specified expression evaluates to TRUE. If multiple statements are to be executed, they must be placed within a BEGIN…END block.

Oracle Microsoft SQL Server
DECLARE
COUNTER NUMBER;
BEGIN
     COUNTER := 0
     WHILE (COUNTER <5) LOOP
          COUNTER := COUNTER + 1;
     END LOOP;
END;

DECLARE
@COUNTER NUMERIC
SELECT@COUNTER = 1
WHILE (@COUNTER <5)
BEGIN
     SELECT @COUNTER =
@COUNTER +1
END

Statement execution can be controlled from inside the loop with the BREAK and CONTINUE keywords. The BREAK keyword causes an unconditional exit from the WHILE loop, and the CONTINUE keyword causes the WHILE loop to restart, skipping any statements that follow. The BREAK keyword is equivalent to the Oracle PL/SQL EXIT keyword. Oracle does not have an equivalent to CONTINUE.

GOTO Statement

Both Oracle and Microsoft SQL Server have GOTO statements, with different syntax. The GOTO statement causes the execution of a Transact-SQL batch to jump to a label. None of the statements between the GOTO statement and the label are executed.

Oracle Microsoft SQL Server
GOTO label;
<<label name here>>

GOTO label
:label

PRINT Statement

The Transact-SQL PRINT statement performs the same operation as the PL/SQL RDBMS_OUTPUT.put_line procedure. It is used for printing user-specified messages. The message limit for the PRINT statement is 8000 characters. Variables that are defined using the char or varchar data type can be embedded in the printed statement. If any other data type is used, the CONVERT or CAST function must be used. Local variables, global variables, and text can be printed. Both single and double quotation marks can be used to enclose text.

Returning from Stored Procedures

Both Microsoft SQL Server and Oracle have RETURN statements. RETURN lets your program exit unconditionally from a query or procedure. RETURN is immediate and complete and can be used at any point to exit from a procedure, batch, or statement block. Statements following RETURN are not executed.

Oracle Microsoft SQL Server
RETURN expression:

RETURN integer_expression

Raising Program Errors

The Transact-SQL RAISERROR statement returns a user-defined error message and sets a system flag to record that an error has occurred. It is similar in function to the PL/SQL raise_application_error exception handler.

The RAISERROR statement allows the client to retrieve an entry from the sysmessages table or build a message dynamically with user-specified severity and state information. When defined, this message is sent back to the client as a server error message.

RAISERROR ({msg_id | msg_str} , severity, state
      [, argument1 [, argument2]])
      [WITH options]

When converting your PL/SQL programs, it may not be necessary to use the RAISERROR statement. In the following code example, the PL/SQL program uses the raise_application_error exception handler, and the Transact-SQL program uses nothing. The raise_application_error exception handler has been included to prevent the PL/SQL program from possibly returning an ambiguous "Unhandled exception" error message. Instead, it always returns the Oracle error message (SQLERRM) when an unanticipated problem occurs.

When a Transact-SQL program fails, it always returns a detailed error message to the client program. Therefore, unless some specialized error handling is required, the RAISERROR statement is not always needed.

Oracle Microsoft SQL Server
CREATE OR REPLACE FUNCTION
DATA.ADD_NAME
(VNAME IN VARCHAR2) RETURN NUMBER AS
BEGIN
DELETE FROM DEPT_ADMIN.DEPT
WHERE DEPT = VDEPT;
RETURN(SQL%ROWCOUNT);
EXCEPTION
WHEN OTHER THEN
RAISE_APPLICATION_ERROR (-20001,SQLERRM);
END DELETE_DEPT;
/

CREATE PROCEDURE
DATA.ADD_NAME
@VNAME VARCHAR(4) AS
DELETE FROM
DEPT_DB.DBO.DEPT
WHERE DEPT = @VDEPT
RETURN @@ROWCOUNT
GO

Implementing Cursors

Oracle always requires that cursors be used with SELECT statements, regardless of the number of rows requested from the database. In Microsoft SQL Server, a SELECT statement that is not enclosed within a cursor returns rows to the client as a default result set. This is an efficient way to return data to a client application. SQL Server provides two interfaces for cursor functions. When cursors are used in Transact-SQL batches or stored procedures, SQL statements can be used to declare, open, and fetch from cursors as well as positioned updates and deletes. When cursors from a DB-Library, ODBC, or OLE DB program are used, the SQL Server client libraries transparently call built-in server functions to handle cursors more efficiently.

When porting a PL/SQL procedure from Oracle, first determine whether cursors are needed to do the same function in Transact-SQL. If the cursor returns only a set of rows to the client application, use a noncursor SELECT statement in Transact-SQL to return a default result set. If the cursor is used to load data a row at a time into local procedure variables, you must use cursors in Transact-SQL.

Table 26.16 shows the syntax for using cursors.

Declaring a Cursor

Although the Transact-SQL DECLARE CURSOR statement does not support the use of cursor arguments, it does support local variables. The values of these local variables are used in the cursor when it is opened. Microsoft SQL Server offers many additional capabilities in its DECLARE CURSOR statement.

The INSENSITIVE option is used to define a cursor that makes a temporary copy of the data to be used by the cursor. All the requests to the cursor are answered by this temporary table. Consequently, modifications made to base tables are not reflected in the data returned by fetches made to this cursor. Data accessed by this type of cursor cannot be modified.

Table 26.16. Syntax for using cursors
Operation | Oracle | Microsoft SQL Server
Declaring a cursor
CURSOR cursor_name
[(cursor_parameter(s))]
IS select_statement;

DECLARE cursor_name
CURSOR
[LOCAL | GLOBAL]
[FORWARD_ONLY | SCROLL]
[STATIC | KEYSET |
DYNAMIC |
FAST_FORWARD]
[READ_ONLY |
SCROLL_LOCKS |
OPTIMISTIC]
[TYPE_WARNING]
FOR select_statement
[FOR UPDATE [OF
column_name [,...n]]]
Opening a cursor
OPEN cursor_name
[(cursor_parameter(s))];
OPEN cursor_name
Fetching from cursor
FETCH cursor_name INTO
variable(s)

FETCH [[NEXT | PRIOR |
FIRST | LAST | ABSOLUTE {n |
@nvar}  | RELATIVE {n | @nvar} ]
FROM] cursor_name
[INTO @variable(s)]
Update fetched row
UPDATE table_name
SET statement(s)...
WHERE CURRENT OF
cursor_name;

UPDATE table_name
SET statement(s)...
WHERE CURRENT OF
cursor_name
Delete fetched row
DELETE FROM table_name
WHERE CURRENT OF
cursor_name;

DELETE FROM table_name
WHERE CURRENT OF
cursor_name;
Closing cursor | CLOSE cursor_name; | CLOSE cursor_name;
Remove cursor data structures | N/A | DEALLOCATE cursor_name

Applications can request a cursor type and then execute a Transact-SQL statement that is not supported by server cursors of the type requested.

SQL Server returns an error that indicates that the cursor type has changed or, given a set of factors, implicitly converts a cursor.

The SCROLL option allows backward, absolute, and relative fetches in addition to forward fetches. A scroll cursor uses a keyset cursor model in which committed deletes and updates made to the underlying tables by any user are reflected in subsequent fetches. This is true only if the cursor is not declared with the INSENSITIVE option.

If the READ ONLY option is chosen, updates are prevented from occurring against any row within the cursor. This option overrides the default capability of a cursor to be updated.The UPDATE [OF column_list] statement is used to define updatable columns within the cursor. If [OF column_list] is supplied, only the columns listed allow modifications. If no list is supplied, all the columns can be updated unless the cursor has been defined as READ ONLY.

It is important to note that the name scope for an SQL Server cursor is the connection itself. This is different from the name scope of a local variable. A second cursor with the same name as an existing cursor on the same user connection cannot be declared until the first cursor is deallocated.

Opening a Cursor

Unlike PL\SQL, Transact-SQL does not support the passing of arguments to a cursor when it is opened. When a Transact-SQL cursor is opened, the result set membership and ordering are fixed. Updates and deletes that have been committed against the base tables of the cursor by other users are reflected in fetches made against all cursors defined without the INSENSITIVE option. In the case of an INSENSITIVE cursor, a temporary table is generated.

Fetching Data

Oracle cursors can move in a forward direction only—there is no backward or relative scrolling capability. SQL Server cursors can scroll forward and backward with the fetch options that follow. These fetch options can be used only when the cursor is declared with the SCROLL option.

NEXT Returns the first row of the result set if this is the first fetch against the cursor; otherwise, it moves the cursor one row within the result set. NEXT is the primary method used to move through a result set. NEXT is the default cursor fetch.

PRIOR Returns the previous row within the result set.

FIRST Moves the cursor to the first row within the result set and returns the first row.

LAST Moves the cursor to the last row within the result set and returns the last row.

ABSOLUTE n Returns the nth row within the result set. If n is a negative value, the returned row is the nth row counting backward from the last row of the result set.

RELATIVE n Returns the nth row after the currently fetched row. If n is a negative value, the returned row is the nth row counting backward from the relative position of the cursor.

The Transact-SQL FETCH statement does not require the INTO clause. If return variables are not specified, the row is automatically returned to the client as a single-row result set. However, if the procedure must get the rows to the client, a noncursor SELECT statement is much more efficient.

The @@FETCH_STATUS function is updated following each FETCH. It is similar in use to the CURSOR_NAME%FOUND and CURSOR_NAME% NOTFOUND variables used in PL/SQL. The @@FETCH_STATUS function is set to the value of 0 following a successful fetch. If the fetch tries to read beyond the end of the cursor, a value of –1 is returned. If the requested row has been deleted from the table after the cursor was opened, the @@FETCH_STATUS function returns –2. The value of –2 usually occurs only in a cursor declared with the SCROLL option. This variable must be checked following each fetch to ensure the validity of the data.

SQL Server does not support Oracle's cursor FOR loop syntax.

CURRENT OF Clause

The CURRENT OF clause syntax and function for updates and deletes is the same in both PL/SQL and Transact-SQL. A positioned UPDATE or DELETE is performed against the current row within the specified cursor.

Closing a Cursor

The Transact-SQL CLOSE CURSOR statement closes the cursor but leaves the data structures accessible for reopening. The PL/SQL CLOSE CURSOR statement closes and releases all data structures. Transact-SQL requires the use of the DEALLOCATE CURSOR statement to remove the cursor data structures. The DEALLOCATE CURSOR statement is different from CLOSE CURSOR in that a closed cursor can be reopened. The DEALLOCATE CURSOR statement releases all data structures associated with the cursor and removes the definition of the cursor.

ODBC

This section provides information about the ways Oracle and SQL Server use ODBC and information about developing or moving applications with ODBC. Use the following process when you convert application code from Oracle to SQL Server:

  1. Consider converting your application to ODBC if it is written using Oracle Pro [image] C or the Oracle Call Interface (OCI).

  2. Understand SQL Server default result sets and cursor options, and choose the fetching strategy that is most efficient for your application.

  3. Remap Oracle ODBC SQL data types to SQL Server ODBC SQL data types where appropriate.

  4. Use the ODBC Extended SQL extensions to create generic SQL statements.

  5. Determine if manual commit mode is required for the SQL Server-based application.

  6. Test the performance of your application(s) and modify the program(s) as necessary.
Forward-only cursors

Oracle and SQL Server treat result sets and cursors differently. Understanding these differences is essential for successfully moving a client application from Oracle to SQL Server and having it perform optimally. In Oracle, any result set from a SELECT command is treated as a forward-only cursor when fetched in the client application, whether you are using ODBC, OCI, or Embedded SQL as the development tool. By default, each Oracle FETCH command issued by the client program (for example, SQLFetch in ODBC) causes a round-trip across the network to the server to return one row. If a client application wants to fetch more than one row at a time across the network, it must set up an array in its program and perform an array fetch.Between fetches, no locks are held at the server for a read-only cursor because of Oracle's multiversioning concurrency model. When the program specifies an updatable cursor with the FOR UPDATE clause, all the requested rows in the SELECT command are locked when the statement is opened. These row-level locks remain in place until the program issues a COMMIT or ROLLBACK request. In SQL Server, a SELECT statement is not always associated with a cursor at the server. By default, SQL Server simply streams all the result set rows from a SELECT statement back to the client. This streaming starts as soon as the SELECT is executed. Result set streams can also be returned by SELECT statements within stored procedures. Additionally, a single stored procedure or batch of commands can stream back multiple result sets in response to a single EXECUTE statement.

The SQL Server client is responsible for fetching default result sets as soon as they are available. For default result sets, fetches at the client do not result in round-trips to the server. Instead, fetches from a default result set pull data from local network buffers into program variables. This default result set model creates an efficient mechanism to return multiple rows of data to the client in a single round-trip across the network. Minimizing network round-trips is usually the most important factor in client/server application performance.

Compared to Oracle's cursors, default result sets put some additional responsibilities on the SQL Server client application. The SQL Server client application must immediately fetch all the result set rows returned by an EXECUTE statement. If the application needs to present rows incrementally to other parts of the program, it must buffer the rows to an internal array. If it fails to fetch all result set rows, the connection to SQL Server remains busy. If this occurs, no other work (such as UPDATE statements) can be executed on that connection until the entire result set rows are fetched or the client cancels the request. Moreover, the server continues to hold share locks on table data pages until the fetch has completed. The fact that these share locks are held until a fetch is complete makes it mandatory to fetch all rows as quickly as possible. This technique is in direct contrast to the incremental style of fetch that is commonly found in Oracle applications.

Server cursors

Microsoft SQL Server offers server cursors to address the need for incremental fetching of result sets across the network. Server cursors can be requested in an application by simply calling SQLSetStmtOption to set the SQL_CURSOR_TYPE option.

When a SELECT statement is executed as a server cursor, only a cursor identifier is returned by the EXECUTE statement. Subsequent fetch requests pass the cursor identifier back to the server along with a parameter specifying the number of rows to fetch at once. The server returns the number of rows requested.

Between fetch requests, the connection remains free to issue other commands, including other cursor OPEN or FETCH requests. In ODBC terms, this means that server cursors allow the SQL Server driver to support multiple active statements on a single connection.

Furthermore, server cursors do not usually hold locks between fetch requests, so you are free to pause between fetches for user input without affecting other users. Server cursors can be updated in place using either optimistic conflict detection or pessimistic scroll locking concurrency options.

Although these features make programming with server cursors more familiar to Oracle developers than using default result sets, they are not free. Compared to default result sets:

  • Server cursors are more expensive in terms of server resources, because temporary storage space is used to maintain cursor state information at the server.

  • Server cursors are more expensive in retrieving a given result set of data because the EXECUTE statement and each fetch request in a server cursor requires a separate round-trip to the server.

  • Server cursors are less flexible in terms of the kind of batches and stored procedures they support. This is because a server cursor can execute only one SELECT statement at a time, whereas default result sets can be used for batches and stored procedures that return multiple result sets or include statements other than SELECT statements.

For these reasons, it is recommended that you limit the use of server cursors to those parts of your application that need their features.

Scrollable cursors

The Oracle RDBMS supports only forward-scrolling cursors. Each row is fetched to the application in the order it was specified in the query. Oracle does not accept requests to move backward to a previously fetched row. The only way to move backward is to close the cursor and reopen it. Unfortunately, you are repositioned back to the first row in the active query set.

Because SQL Server supports scrollable cursors, you can position an SQL Server cursor at any row location. You can scroll both forward and backward. For many applications involving a user interface, scrollability is a useful feature. With scrollable cursors, an application can fetch a screen full of rows at a time and fetch additional rows only as the user asks for them.

Although Oracle does not directly support scrollable cursors, this limitation can be minimized by using one of several ODBC options. For example, some Oracle ODBC drivers, such as the one that ships with the Microsoft Developer Studio visual development system, offer client-based scrollable cursors in the driver itself. Alternatively, the ODBC Cursor Library supports block scrollable cursors for any ODBC driver that complies with Level One conformance. Both these client cursor options support scrolling by using the RDBMS for forward-only fetching and by caching result set data in memory or on disk. When data is requested, the driver retrieves it from the RDBMS or its local cache as needed.

Client-based cursors also support positioned UPDATE and DELETE statements for the result sets generated by SELECT statements. The cursor library constructs an UPDATE or DELETE statement with a WHERE clause that specifies the cached value for each column in a row.

If you need scrollable cursors and are trying to maintain the same source code for both Oracle and SQL Server implementations, the ODBC Cursor Library is a useful option. For more information about the ODBC Cursor Library, see the ODBC documentation.

Strategies for using SQL server default result sets and server cursors

With all the options that SQL Server offers for fetching data, it is sometimes difficult to decide what to use and when. Here are some useful guidelines.

  • Default result sets are always the fastest way to get an entire set of data from SQL Server to the client. Look for opportunities in your application where you can use this to your advantage. Batch report generation, for example, generally processes an entire result set to completion with no user interaction and no updates in the middle of processing.

  • If a program requires updatable cursors, use server cursors. Default result sets are never updatable when using positioned UPDATE or DELETE statements. Additionally, server cursors are better at updating than client-based cursors, which have to simulate a positioned UPDATE or DELETE by constructing an equivalent searched UPDATE or DELETE statement.

  • If a program needs scrollable, read-only cursors, both the ODBC Cursor Library and server cursors are good choices. The ODBC Cursor Library gives you compatible behavior across SQL Server and Oracle, and server cursors give you more flexibility as to how much data to fetch across the network at one time.

  • When you use default result sets or ODBC Cursor Library cursors built on top of default result sets, be sure to fetch to the end of a result set as quickly as possible to avoid holding share locks at the server.

  • When you use server cursors, be sure to use SQLExtendedFetch to fetch in blocks of rows rather than a single row at a time. This is the same as array-type fetching in Oracle applications. Every fetch request on a server cursor requires a round-trip from the application to the RDBMS on the network.

  • Shopping provides an analogy. Assume you purchase several bags of goods, load one bag into your car, drive home, drop it off, and return for the next bag. This is an unlikely scenario, but this is what you do to SQL Server and your program by making single-row fetches from a server cursor.

  • If your program requires only forward-only, read-only cursors but depends on multiple open cursors on the same connection, use default result sets when you know you can fetch the entire result set immediately into program variables. Use server cursors when you do not know if you can fetch all of the rows immediately.

  • For singleton fetches, using a default result set is more efficient than using a server cursor.


Developing and Administering Database Replication

This section explains the differences between Oracle and Microsoft SQL Server replication support.

Oracle | Microsoft SQL Server
Read-only snapshot | Snapshot with immediate updating subscribers
Replication
Multimaster replication, Updatable snapshot

Transactional replication with immediate updating subscribers—better for well-connected subscribers who are online when updating.

Merge Replication—better for mobile disconnected. This supports default and custom conflict resolution.

As its name implies, SQL Server snapshot replication takes a picture, or snapshot, of the published data in the database at a moment in time. Snapshot replication requires less constant processor overhead than transactional replication because it does not require continuous monitoring of data changes on source servers. Instead of copying INSERT, UPDATE, and DELETE statements (characteristic of transactional replication) or data modifications (characteristic of merge replication), Subscribers are updated by a total refresh of the data set. Hence, snapshot replication sends all the data to the Subscriber instead of sending only the changes. SQL Server also offers transactional replication, a type of replication that marks selected transactions in the Publisher's database transaction log for replication and then distributes them asynchronously to Subscribers as incremental changes, while maintaining transactional consistency. SQL Server merge replication allows sites to make autonomous changes to replicated data and at a later time to merge changes made at all sites. Like Oracle, merge replication supports both column-level and row-level conflict detection. That is, a conflict can be defined to be any change to the same row at two locations or only when changes to the same column(s) at two locations. SQL Server offers heterogeneous replication, which is the simplest way to publish data to a heterogeneous Subscriber by using ODBC or OLE/DB and creating a push subscription from the Publisher to the ODBC Subscriber. As an alternative, however, you can create a publication and then create an application with an embedded distribution control. The embedded control implements the pull subscription from the Subscriber to the Publisher. For ODBC Subscribers, the subscribing database has no administrative capabilities regarding the replication being performed. Following are the conflict-resolution mechanisms for Oracle and SQL Server:

Oracle | Microsoft SQL Server
Site priority or priority value resolvers programmed using PL/SQL | Priority-based resolution using COM or Transact-SQL
Conflict resolution for column groups | Supported through custom resolvers
Column-level and row-level conflict resolution | Support for both


Moving Data and Applications

The simplest way to move data and applications from Oracle to Microsoft SQL Server 2000 is to use the Data Transformation Services (DTS) feature in the SQL Server. The DTS Import/Export Wizard guides you through moving the data to SQL Server. Alternatively, you can create your own DTS package in the DTS designer. You should first disable the constraints in the SQL Server database, then copy over the data, and then enable the constraints. A preferable way is to create the schema using your own scripts (from the data-modeling tool) rather than allowing DTS to create the schema for you, since it can make wrong choices in mapping the data types for you. Then once you have the schema in place, you can disable the constraints using the USP_Disable_All script (code in Chapter 27), then copy over the data using the CopyData task in the DTS designer, and then enable the constraints using the USP_Enable_All script (code in Chapter 27). (DTS is discussed in Chapter 5).

DTS was designed to move and transorm data from multiple sources. If you are moving tables and data from an Oracle database, it works pretty well. However, without the capability to parse and translate PL/SQL code into transact-SQL, DTS is only partially useful.


Summary

In this chapter, we compared the features of SQL Server 2000 and Oracle 9i. We also discussed various pitfalls that can take place in moving Oracle databases to SQL Server, and we discussed how to move the data and the schema as well.


Part IV: Scripts and Utilities

In this part, we will go over some administration and utility scripts that can help the DBAs and the database developers in doing the day-to-day administration work in a cost-effective manner. The code for these utilities is also on the CD-ROM for the book.


Chapter 27. Administration and Utility Scripts

This chapter presents some of the administrative and utility scripts that could make the day-to-day administration and management of the SQL Server databases easy. The sample scripts are in T-SQL code and have been prefixed with USP_, indicating that they are user-created stored procedures, not the system procedures, which begin with SP_.


Scripts

Disable Constraints

This script (in T-SQL code) can be used for disabling all the check and foreign key constraints in a given database. You can create this procedure in the database and then execute it to disable the constraints, or you can use the body of the stored procedure and run it as is. The procedure builds up a temporary table with a list of all the check and foreign key constraints in the database and then loops through them using a while loop and disables the constraints.

/*
DISABLE ALL TABLE CONSTRAINTS

This script disables all constraints on all tables within the database in which it
is run.

*/

CREATE PROC USP_DISABLE_ALL
AS
BEGIN
SET NOCOUNT ON
SET ROWCOUNT 0

DECLARE @Count int
DECLARE @String nvarchar (1000)
DECLARE @ConstraintName varchar(128)
DECLARE @TableName varchar(128)

if exists (select name from tempdb..sysobjects where name like '#MyTable%')
BEGIN
     DROP TABLE #MyTable
END

/*Find all constraints and their respective tables from the sysobjects table and
place into a temp table. NOCHECK can be used only for the foreign key and check
constraints, not for the primary key or unique constraints and also not for the
defaults.
*/
SELECT name AS constraintname,
     (select name from sysobjects where id = s.parent_obj) AS tablename
INTO #MyTable
FROM sysobjects s where xtype in ('C', 'F')

SELECT @Count = Count(*) FROM #MyTable

-- Setting the rowcount to 1 allows for one row from the temp table to be picked --
-- off at a time.
-- Used as an alternative to a cursor.
SET ROWCOUNT 1

-- Loop until all rows in temp table have been processed.
WHILE @Count > 0
BEGIN
     -- The rowcount of 1 ensures that only one tablename and constraint name is
     -- picked.
     SELECT @TableName = TableName, @ConstraintName = ConstraintName
     FROM #MyTable

     -- Build execution string to disable constraint.
     SET @String = 'ALTER TABLE ['+ @tablename + '] NOCHECK CONSTRAINT [' +
         @constraintname +']'

     exec sp_executesql @string
     -- Remove this row from the temp table, since it has now been processed.
     DELETE FROM #MyTable WHERE ConstraintName = @ConstraintName and TableName =
     @TableName
     SET @Count = @Count - 1
END -- Loop

SET ROWCOUNT 0
END
GO

Enable Constraints

This script (in T-SQL code) is a supplement to the previous one and can be used for enabling all the check and foreign key constraints in a given database. You can either create this procedure in the database and then execute it, or you can use the body of the stored procedure and run it as is. The procedure builds up a temporary table with a list of all the check and foreign key constraints in the database and then loops through them using a while loop and enables the constraints.

/*
ENABLE ALL TABLE CONSTRAINTS
This script enables all constraints on all tables within the database in which it
is run.
*/

CREATE PROC USP_ENABLE_ALL
AS
BEGIN
SET NOCOUNT ON
SET ROWCOUNT 0

DECLARE @Count int
DECLARE @String nvarchar (1000)
DECLARE @ConstraintName varchar(128)
DECLARE @TableName varchar(128)

if exists (select name from tempdb..sysobjects where name like '#MyTable%')
BEGIN
     DROP TABLE #MyTable
END
/*Find all constraints and their respective tables from the sysobjects table and
place into a temp table. CHECK can be used only for the foreign key and check
constraints, not for the primary key or unique constraints and also not for the
defaults.
*/
SELECT name AS constraintname,
      (select name from sysobjects where id = s.parent_obj) AS tablename
INTO #MyTable
FROM sysobjects s where xtype in ('C', 'F')

SELECT @Count = Count(*) FROM #MyTable

-- Setting the rowcount to 1 allows for one row from the temp table to be picked
-- off at a time.
-- Used in conjunction with WHILE Loop as an alternative to a cursor.
SET ROWCOUNT 1

-- Loop until all rows in temp table have been processed.
WHILE @Count > 0
BEGIN
     -- The rowcount of 1 ensures that only one tablename and constraint name is
picked.
     SELECT @TableName = TableName, @ConstraintName = ConstraintName
     FROM #MyTable

     -- Build execution string to enable constraint.
     SET @String = 'ALTER TABLE ['+ @tablename + '] CHECK CONSTRAINT [' +
@constraintname
      +']'

     exec sp_executesql @string

     -- Remove this row from the temp table, since it has now been processed.
     DELETE FROM #MyTable WHERE ConstraintName = @ConstraintName and TableName =
     @TableName

     SET @Count = @Count - 1
END -- Loop

SET ROWCOUNT 0
END
GO

USP_Find_Lock

This script (in T-SQL code) can be used for finding out the blocking SPIDs and the blocked SPIDs in the database. The script creates a table named DEBUG_LOCK in the database, which tracks the blocker and the blocking SPIDs, the SQL statements they were executing at the time of the block, and the amount of time that the wait has occurred for the blocked SPIDs. This stored procedure can be run whenever you want to check the status of the locks to make sure that there aren't any blocked processes in your system, or you can schedule the procedure to be run every 10 to 15 minutes so that you can get a report on the sequence of events leading up to the blocking scenario.

if (select count(*) from sysobjects where name = 'Debug_Lock') = 1
begin
drop table DEBUG_LOCK
end
go
CREATE TABLE DEBUG_LOCK
(
Debug_Lock_ID int identity,
Blocker_Spid int,
Blocker_SQL varchar(2500),
Blockee_Spid int,
Blockee_SQL varchar(2500),
Blockee_Waittime int,
create_date_time datetime
)
GO
if (select count(*) from sysobjects where name = 'USP_Find_lock') = 1
begin
drop proc USP_Find_lock
end
go
CREATE PROCEDURE USP_Find_lock
AS

set nocount on

declare @blocker_spid int, @blockee_spid int, @blockee_blocker int
declare @blockee_waitime int

IF EXISTS
    (SELECT * FROM master.dbo.sysprocesses
    WHERE spid IN (SELECT blocked FROM master.dbo.sysprocesses))
    BEGIN

DECLARE blocker_cursor CURSOR FOR
     SELECT spid FROM master.dbo.sysprocesses WHERE spid IN (SELECT blocked FROM
     master.dbo.sysprocesses) AND blocked=0

DECLARE blockee_cursor CURSOR FOR
     SELECT spid, blocked, waittime FROM master.dbo.sysprocesses WHERE blocked > 0

create table #blockertemp (eventtype varchar(20), Parameters varchar(20), EventInfo
varchar(1000))
create table #blockeetemp (eventtype varchar(20), Parameters varchar(20), EventInfo
varchar(1000))

OPEN blocker_cursor

FETCH NEXT FROM blocker_cursor INTO @blocker_spid
     WHILE (@@FETCH_STATUS =0 )
          BEGIN
               Select 'Blocker: ',@blocker_spid
               exec sp_executesql N'dbcc inputbuffer(@Param)',N'@Param int',
@blocker_spid
               insert into #blockertemp exec sp_executesql N'dbcc
               inputbuffer(@Param)',N'@Param int', @blocker_spid

               OPEN blockee_cursor
                         FETCH NEXT FROM blockee_cursor INTO @blockee_spid,
                         @blockee_blocker, @blockee_waitime

                         WHILE (@@fetch_status = 0)
                              BEGIN
                              SELECT 'Blockee:    Waittime:', @blockee_spid,
                              @blockee_waitime
                              insert into #blockeetemp exec sp_executesql N'dbcc
                              inputbuffer(@Param)',N'@Param int', @blockee_spid
                              insert into Debug_Lock (blocker_spid, blocker_sql,
                              blockee_spid, blockee_sql, blockee_waittime,
                              create_date_time)
                              select @blockee_blocker, (select eventinfo from
                              #blockertemp), @blockee_spid, eventinfo,
@blockee_waitime,
                              getdate() from #blockeetemp
                              delete from #blockeetemp
                              FETCH NEXT FROM blockee_cursor INTO @blockee_spid,
                              @blockee_blocker, @blockee_waitime
                              END
               CLOSE blockee_cursor
          FETCH NEXT FROM blocker_cursor INTO @blocker_spid
          END
CLOSE blocker_cursor
DEALLOCATE blockee_cursor
DEALLOCATE blocker_cursor

    END
ELSE
SELECT 'No blocking processes found!'
go
exec USP_Find_lock
go
select * from debug_lock

USP_Lock_Info

The last procedure gave information only on the blocks. If you want to find out the locks being held in the database at any given time, then you can use the following procedure (in T-SQL code). It basically joins the sp_lock procedure information with the information from the sysprocesses table and displays it to the end user.

CREATE PROCEDURE USP_lock_Info
WITH ENCRYPTION
AS
begin
create table #temp (spid int, dbid int, objid bigint, Indid int, type varchar(10),
resource varchar(30), mode varchar(10), status varchar(20))
insert into #temp exec sp_lock
select #temp.spid as spid, sp.hostprocess, db_name (#temp.dbid) as db_name,
object_name (#temp.objid) object_name, #temp.Mode,
sp.login_time, sp.last_batch, sp.open_tran, sp.status, sp.hostname,
sp.program_name,
sp.cmd, sp.loginame
from #temp, sysprocesses sp
where #temp.spid = sp.spid
and #temp.objid <> 0
drop table #temp
end
go

Greatest

This function is a substitute for the Greatest function provided by Oracle. This takes in two parameters and returns the one with the higher value to the calling program. It uses the new data type sql_variant for the local variables since we don't know what kind of values will be passed in to the function.

CREATE FUNCTION DBO.GREATEST
    ( @PARAMETER1 sql_variant, @PARAMETER2 sql_variant )

RETURNS sql_variant
AS
BEGIN
DECLARE @GREATEST sql_variant
IF (@PARAMETER1 > @PARAMETER2)
SET @GREATEST = @PARAMETER1
ELSE SET @GREATEST = @PARAMETER2
    RETURN @GREATEST
END
GO

Update Statistics

For production databases, it is never a good idea to leave the auto-stats option on, because the query optimizer checks at frequent intervals whether it needs to update the statistics and that takes a performance hit. A good idea instead is to schedule a task for updating the statistics every night at a time when transaction volume is low.

sp_MSforeachtable
@replacechar='~',
@precommand="print 'Updating Statistics.....' print "",
@command1="print '~' update statistics ~",
@postcommand="print" print 'Complete Updating Statistics!'"

You could also use just the command sp_updatestats in a given database.

USP_Active

This procedure (in T-SQL code) can be used for finding out which users are active in the system at any given time. It excludes the system processes from the list and also gives a list of the sql statements being executed by those active processes.

CREATE PROCEDURE USP_Active
AS
SET NOCOUNT ON
DECLARE @spid int

/**** Get statistical process information ***/
SELECT
     spid,
     blocked,
     'Database' = CASE WHEN master.dbo.sysdatabases.name IS NULL THEN " ELSE
     CAST(master.dbo.sysdatabases.name AS char(15)) END,
     cpu,
     physical_io,
     'program_name' = CAST(program_name AS char(15)),
     cmd,
     loginame
FROM
     master.dbo.sysprocesses
     left outer join
     master.dbo.sysdatabases
     on master.dbo.sysdatabases.dbid = master.dbo.sysprocesses.dbid
WHERE
     cmd <> 'awaiting command'   /** Inactive processes are excluded ***/
AND     spid > 11  /*** System processes are excluded ***/

/*** Use cursor to get SQL Statement for each connection ***/

DECLARE @sqlstring varchar(250)

CREATE TABLE #temp (col1 nvarchar(30), col2 int, col3 nvarchar(255))

DECLARE cur CURSOR FOR
               SELECT     spid
               FROM
                    master.dbo.sysprocesses
                    left outer join
                    master.dbo.sysdatabases
                    on master.dbo.sysdatabases.dbid = master.dbo.sysprocesses.dbid
               WHERE
                    cmd <> 'awaiting command'
               AND     spid >11
OPEN cur
FETCH NEXT FROM cur INTO @spid
WHILE @@fetch_status = 0
BEGIN
     SET @sqlstring = 'DBCC INPUTBUFFER (' + CAST(@spid AS CHAR(3)) + ')'
     INSERT #temp (col1, col2, col3)
     EXEC (@sqlstring)

     FETCH NEXT FROM cur INTO @spid
END
CLOSE cur
DEALLOCATE cur

SELECT COL3 AS SQL_Statement
FROM #temp

DROP TABLE #temp
SET NOCOUNT OFF
GO

Objects

This script (in T-SQL code) can be used for detailing the objects in the database. It gives a count of all the tables, views, stored procedures, triggers, and so on in the database after querying the sysobjects table in the database. You can change it to give the name of the objects if you so desire; just change the count( [image]) to name.

print 'Number of primary keys'

select count(*) from sysobjects where xtype = 'PK'

print 'Number of Indexes'

select count(*)
from sysindexes
inner join
sysobjects
on sysobjects.id = sysindexes.id
where sign(sysobjects.status) = 1
and sysindexes.name not like '_WA%'     /*to eliminate the temp index list (for
internal use)created by sql server based on statistics*/
order by 1

print 'Number of defaults'

select count(*) from sysobjects where xtype = 'D'

print 'Number of views'

select count(*) from sysobjects where xtype = 'v' and sign(status) = 1

print 'Number of stored procedures'

select count(*) from sysobjects where xtype = 'p' and sign(status) = 1

print 'Number of triggers'

select count(*) from sysobjects where xtype = 'tr'

Number_Of_Rows

If you want to find out the number of rows in each table in the database, you can use one of these three Transact-SQL scripts to achieve this.

  1. The results of this SQL depend on whether the statistics have been kept up to date with the passage of time because it uses the sysindexes table to get the total count of records.

    select     substring(o.name, 1, 30) Table_Name ,i.rows  Number_of_Rows
from       sysobjects o
inner join sysindexes i
on (o.id = i.id)
where      o.xtype = 'u'
and        i.indid < 2
order by o.name

    The xtype = 'u' is for selecting the tables and indid < 2 makes sure that we are selecting all the tables, whether they do or do not have indexes on them.

  2. This SQL can also be used to count the number of records in the tables in a database. It uses the count aggregate function to do so. Selecting the table names and putting quotes around them is important for taking care of the spaces in the names of the tables, if any.

    SET NOCOUNT ON
   DECLARE tables_cursor CURSOR
      FOR
      SELECT name FROM sysobjects WHERE type = 'U' order by 1
   OPEN tables_cursor
   DECLARE @tablename varchar(50)

   FETCH NEXT FROM tables_cursor INTO @tablename

   WHILE (@@fetch_status <> -1)
   BEGIN
   select @tablename = quotename(@tablename, '[]')
     DECLARE @CMD VARCHAR(100)
     PRINT @TABLENAME
     SELECT @CMD = 'select count(*) from  '+@tablename+''
     EXEC (@CMD)

      FETCH NEXT FROM tables_cursor INTO @tablename
   END
   DEALLOCATE tables_cursor
   SET NOCOUNT OFF

  3. The undocumented stored proc sp_Msforeachtable also achieves the goal. Sp_MSforeachtable scans through all the tables in a certain database and performs a command on them. A question mark (?) is used as a placeholder to represent the table name.

    sp_msforeachtable @command1= 'print "?'", @command2 = "select count(*) from ?"

USP_Update_One_Table

This script (in T-SQL code) can be used for updating a single table from one database to the other. An easy way to do it is to disable the constraints in the destination database, delete the data from the destination table, and then enable the constraints. This procedure expects the USP_Disable_All and the USP_Enable_All procedures shown earlier in this chapter to be present in the destination database.

Note

If you have identity columns, this script will fail, since for an identity column table, you need to first set the identity_insert to on and then you need to specify the list of the columns for the entire table.

/*
USP_Update_One_Table:

This script can be used for copying one table from the source database to the
destination database.
Input parameters: SourceDatabase, DestinationDatabase, and the name of the table
that needs to be copied.
Prerequisites: The USP_DISABLE_ALL and USP_ENABLE_ALL stored procedures should
already exist in the destination database.
*/

Create proc USP_UPDATE_ONE_TABLE
@SourceDatabase varchar(50),
@DestDatabase varchar(50),
@table_name varchar(50)
with encryption
as
begin
declare @command1 varchar(100), @command2 varchar(100), @command3 varchar(5000),
@command4 varchar(100)
select @command1 = "+@DestDatabase+'.dbo.USP_disable_all'
select @command2 = 'delete from ' +@DestDatabase+'.dbo.'+@Table_Name+"
select @command3 = 'insert into ' +@DestDatabase+'.dbo.'+@Table_Name+' select *
from ' +@SourceDatabase+'.dbo.'+@Table_Name+"
select @command4 = "+@DestDatabase+'.dbo.USP_enable_all'

exec (@command1)
exec (@command2)
exec (@command3)
exec (@command4)

end
go

USP_System_Info

This procedure (in T-SQL code) can be used for giving the system information and for finding the different options selected at the time of the installation of the database. It uses the SERVERPROPERTY function of SQL Server 2000.

CREATE PROC USP_SYSTEMINFO
AS
BEGIN
SELECT   CONVERT(char(50), SERVERPROPERTY('COLLATION')) AS COLLATION,
     CONVERT(char(20), SERVERPROPERTY('EDITION')) AS EDITION,

     (CASE WHEN CONVERT(char(20), SERVERPROPERTY('InstanceName')) IS NULL
             THEN 'DEFAULT INSTANCE'
           ELSE CONVERT(char(20), SERVERPROPERTY('InstanceName'))
     END) AS Instance_Name,

     (CASE WHEN CONVERT(char(20), SERVERPROPERTY('ISClustered')) = 1
             THEN 'CLUSTERED'
           WHEN CONVERT(char(20), SERVERPROPERTY('ISClustered')) = 0
             THEN 'NOT CLUSTERED'
           ELSE 'INVALID INPUT/ERROR'
     END) AS FAILOVER_CLUSTERED,

     (CASE WHEN CONVERT(char(20), SERVERPROPERTY('ISFullTextInstalled')) = 1
             THEN 'Full Text - Installed'
           WHEN CONVERT(char(20), SERVERPROPERTY('ISFulltextInstalled')) = 0
             THEN 'Full Text - NOT Installed'
           ELSE 'INVALID INPUT/ERROR'
     END) AS FULL_TEXT_INSTALLATION,

     (CASE WHEN CONVERT(char(20), SERVERPROPERTY('ISIntegratedSecurityOnly')) = 1
             THEN 'Integrated Security'
           WHEN CONVERT(char(20), SERVERPROPERTY('ISIntegratedSecurityOnly')) = 0
             THEN 'SQL Server Security'
           ELSE 'INVALID INPUT/ERROR'
     END) AS SECURITY,

     (CASE WHEN CONVERT(char(20), SERVERPROPERTY('ISSingleUser')) = 1
             THEN 'Single User'
           WHEN CONVERT(char(20), SERVERPROPERTY('ISSingleUser')) = 0
             THEN 'Multi User'
           ELSE 'INVALID INPUT/ERROR'
     END) AS Single_User,

     (CASE WHEN CONVERT(char(20), SERVERPROPERTY('LicenseType')) = 'PER_SEAT'
             THEN 'Per Seat Mode'
           WHEN CONVERT(char(20), SERVERPROPERTY('LicenseType')) = 'PER_PROCESSOR'
             THEN 'Per Processor Mode'
           ELSE 'Disabled'
     END) AS License_Type,

     CONVERT(char(20), SERVERPROPERTY('MachineName')) AS Machine_Name,
     CONVERT(char(20), SERVERPROPERTY('NumLicenses')) AS Number_Of_Licenses,

/*To identify which sqlservr.exe belongs to this instance*/
     CONVERT(char(20), SERVERPROPERTY('ProcessID')) AS Process_ID,
     /*The version of SQL Server instance in the form: major.minor.build*/
     CONVERT(char(20), SERVERPROPERTY('ProductVersion')) AS Product_Version,
     /*Level of the version of SQL Server Instance*/
     CONVERT(char(20), SERVERPROPERTY('ProductLevel')) AS Product_Level,
     CONVERT(char(20), SERVERPROPERTY('ServerName')) AS Server_Name
END
GO

USP_Null_Columns

This script (in T-SQL code) can be used for finding out which columns allow null values in the database.

SELECT o.name [Table Name],
c.name [Column Name],
case c.isnullable
when 0 then 'not null'
else 'allows null'
end  [NULL/NOT NULL]
FROM sysobjects o
left outer join
syscolumns c
on o.id = c.id

WHERE o.type = 'u'
order by 1

USP_SetUP_Paths

This script (in T-SQL code) can be used to get the data, tools, and SQL path when there are several instances on the same server. The system SP used in this script is undocumented, so use it at your own risk.

USE MASTER
go

DECLARE @DataPath nvarchar( 512 ) , @SQLPath nvarchar( 512 ) ,
        @ToolsPath nvarchar( 512 ) , @CmdStr nvarchar( 4000 );
EXECUTE dbo.sp_MSgettools_path @ToolsPath OUT;
EXECUTE sp_MSget_setup_paths @SQLPath OUT , @DataPath OUT;
PRINT 'SQLPath   : ' + QUOTENAME( @SQLPath , "");
PRINT 'DataPath  : ' + QUOTENAME( @DataPath , "");
PRINT 'ToolsPath : ' + QUOTENAME( @ToolsPath , "");
SET @CmdStr = QUOTENAME( @ToolsPath + '\BINN\OSQL', "") + ' -L'
PRINT 'CMDStr    : ' + @CmdStr
EXECUTE master..xp_cmdshell @CmdStr
GO

USP_Identity_Columns

This script (in T-SQL code) can be used for finding the identity columns in the tables in a database.

SELECT
  IDENT_SEED(TABLE_NAME) AS Seed
, IDENT_INCR(TABLE_NAME) AS Increment
, TABLE_NAME
 FROM INFORMATION_SCHEMA.TABLES
 WHERE OBJECTPROPERTY(OBJECT_ID(TABLE_NAME), 'TableHasIdentity') = 1
 AND TABLE_TYPE = 'BASE TABLE'

USP_DB_File_Info

This script (in T-SQL code) displays name, size, location, and filegroups information for all the databases created within SQL Server.

DECLARE @dbname varchar(50)
DECLARE @command varchar(255)
DECLARE dbname_cursor CURSOR FOR SELECT name from master..sysdatabases where name
not in ('northwind', 'pubs')
OPEN dbname_cursor
FETCH next FROM dbname_cursor into @dbname
WHILE @@fetch_status = 0
BEGIN
SELECT @command = 'USE ' + @dbname + ' select ' + 'convert(varchar(25),f.name)
"Device Name'','
+ 'convert(varchar(10),size/128) + " MB" "Device Size",'
+ 'convert(varchar(100),f.filename) Path,'
+ 'convert(varchar(15),filegroup_name(groupid)) Filegroup from sysfiles f'
EXEC (@command)
FETCH NEXT FROM dbname_cursor INTO @dbname
END
CLOSE dbname_cursor
DEALLOCATE dbname_cursor
GO

USP_NOIndexes

This script (in T-SQL code) can be used to find out the names of the tables that do not have any indexes and to display the number of records in them.

select sysobjects.name Table_Name,
(Select rows from sysindexes
 where id = sysobjects.id
 and indid = 0) Rows
from
 sysobjects
where xtype = 'u'
 and objectproperty(sysobjects.id, 'TableHasIndex') = 0

USP_RenameDB

This script (in T-SQL code) can be used for renaming a database on the database server.

-- This script can be used to rename the databases.
-- Input parameters: database names: old and new.
-- Output parameter: None.

CREATE PROCEDURE USP_RenameDB
    @OldDatabaseName varchar(30),
    @NewDatabaseName varchar(30)
WITH ENCRYPTION
AS
BEGIN
    DECLARE @OldTranCount  INTEGER;

    SET @OldTranCount = @@TRANCOUNT;

    WHILE (@@TRANCOUNT > 0)
        ROLLBACK TRANSACTION

    EXEC sp_dboption @OldDatabaseName, 'single user', 'TRUE'
    EXEC sp_renamedb @OldDatabaseName, @NewDatabaseName
    EXEC sp_dboption @NewDatabaseName, 'single user', 'FALSE'

    WHILE (@@TRANCOUNT < @OldTranCount)
        BEGIN TRANSACTION
END
GO

USP_ForceDropDB

This script (in T-SQL code) drops a given database even if there are any connections to the database. It kills the connections first and then drops the database.

-- This script drops a database even if there are connections by killing the
-- connections.
-- Input parameters: File path for the database.
-- Output Parameter: None.
CREATE PROC USP_ForceDropDB
    @DatabaseName varchar(100)
WITH ENCRYPTION
AS
BEGIN
    -- Cursor for all the spids running against this database
    DECLARE SysProc CURSOR LOCAL FORWARD_ONLY DYNAMIC READ_ONLY FOR
        SELECT spid
        FROM master.dbo.sysprocesses
        WHERE dbid =
         (SELECT dbid FROM master.dbo.sysdatabases
          WHERE name = @DatabaseName)
    DECLARE @SysProcId smallint
    OPEN SysProc    -- kill all the processes running against the database
    FETCH NEXT FROM SysProc INTO @SysProcId
    DECLARE @KillStatement char(40)
    WHILE @@FETCH_STATUS = 0
    BEGIN
        SET @KillStatement = 'KILL ' + CAST(@SysProcId AS char(30))
        EXEC (@KillStatement)
        FETCH NEXT FROM SysProc INTO @SysProcId
    END
        WAITFOR DELAY '000:00:01'
        DROP DATABASE @DatabaseName
END
GO

USP_Trace_BlackBox

This script (in T-SQL code) can be used for utilizing the black-box trace functionality (the flight recorder utility) of SQL Server 2000. It creates a trace file ( [image].trc) that can later be viewed through the profiler and analyzed.

if exists (select * from dbo.sysobjects where id =
object_id(N'[dbo].[trace_blackbox]') and OBJECTPROPERTY(id, N'IsProcedure') = 1)
drop procedure [dbo].[USP_trace_blackbox]
GO

SET QUOTED_IDENTIFIER ON
GO
SET ANSI_NULLS ON
GO
/*****************************************************************************
Usage:

USP_trace_blackbox: Stored procedure for creating the black box trace (the flight
recorder utility)
USP_trace_blackbox 1 starts the trace.
USP_trace_blackbox 0 stops the trace.
USP_trace_blackbox gives the current status of the trace.
*****************************************************************************/

CREATE PROCEDURE USP_trace_blackbox @on int = 2
AS

/* If no argument is passed to the @on parameter then get the current black box
trace status.
   If @on is 0, then stop and delete the blackbox trace.
   If @on is 1, then create and start the blackbox trace.
*/

declare @traceid int, @blackboxstatus int, @dir nvarchar(80)
set @traceid = 0
set @blackboxstatus = 0
set nocount on
SELECT @traceid = traceid FROM :: fn_trace_getinfo(0)
 where property = 1 and value = 8

IF @on = 0 and @traceid > 0
 begin
  select @blackboxstatus = cast(value as int) FROM :: fn_trace_getinfo(0)
   where traceid = @traceid and property = 5
  IF @blackboxstatus > 0 exec sp_trace_setstatus @traceid,0 -- stop blackbox trace
  exec sp_trace_setstatus @traceid,2 -- delete blackbox trace definition
 end

IF @on = 1
   begin
    IF @traceid < 1 exec sp_trace_create @traceid OUTPUT, 8 -- create blackbox
    -- trace
    exec sp_trace_setstatus @traceid,1 -- start blackbox trace
   end

set @traceid = 0
set @blackboxstatus = 0
SELECT @traceid = traceid FROM :: fn_trace_getinfo(0)
 where property = 1 and value = 8
select @blackboxstatus = cast(value as int) FROM :: fn_trace_getinfo(0)
 where traceid = @traceid and property = 5
IF @traceid > 0 and @blackboxstatus > 0
   begin
    select @dir = cast(value as nvarchar(80)) FROM :: fn_trace_getinfo(0)
     where traceid = @traceid and property = 2
    select 'The blackbox trace is running and the trace file is in the following
directory.'
    select @dir + '.trc'
   end
ELSE select 'The blackbox trace is not running.'

set nocount off

GO
SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS ON
GO

Registry_Info

This procedure (in T-SQL code) can be used for getting the information from the registry through Transact-SQL using the OLE Automation stored procedures.

USE master
GO

CREATE PROCEDURE USP_registry_info (
        @ServerName varchar( 128 ) = NULL ,
        @UserName varchar( 128 ) = '' ,
        @Password varchar( 128 ) = ''
)
AS
SET NOCOUNT ON
-- Table for storing and displaying the values.
CREATE TABLE #Registry(
        Property varchar( 30 ) , Value varchar( 255 )
)
DECLARE @SQLServer int , @Registry int , @Property int , @PropCount int ,
        @Name varchar( 30 ) , @Value varchar( 255 ) , @RetCode int , @Counter int
SELECT @Counter = 1 , @Value = '' ,
       -- Default to local server if no server name is passed.
       @ServerName = COALESCE( @ServerName , @@SERVERNAME )

EXEC @RetCode = sp_OACreate 'SQLDMO.SQLServer' , @SQLServer OUT
IF @RetCode <> 0 Or @@ERROR <> 0 Or @SQLServer < 0 GOTO Error_Handler

-- Use trusted connection if user name is blank.
IF @UserName = ''
        EXEC sp_OASetProperty @SQLServer, 'LoginSecure' , 1

-- Connect to the server.
EXEC @RetCode = sp_OAMethod @SQLServer , 'Connect' , NULL , @ServerName , @UserName
, @Password
IF @RetCode <> 0 Or @@ERROR <> 0 GOTO Error_Handler

-- Obtain the registry object.
EXEC sp_OAGetProperty @SQLServer , 'Registry' , @Registry OUT
-- Get the total number of properties for the registry object.
EXEC sp_OAGetProperty @Registry  , 'Properties.Count' , @PropCount OUT

WHILE ( @Counter <= @PropCount )
BEGIN
        -- Obtain each property from the registry.
        EXEC sp_OAGetProperty @Registry , 'Properties' , @Property OUT , @Counter

        -- Get name and value of the property.
        EXEC sp_OAGetProperty @Property , 'Name'       , @Name OUT
        EXEC sp_OAGetProperty @Property , 'Value'      , @Value OUT

        INSERT INTO #Registry (
                Property , Value
        )
        VALUES( @Name , @Value )

        SELECT @Counter = @Counter + 1 , @Value = ''
END

SELECT r.Property , r.Value
  FROM #Registry AS r
 ORDER BY r.Property
GOTO Cleanup

Error_Handler:
        EXEC sp_OAGetErrorInfo @SQLServer , @RetCode

Cleanup:
        IF @SQLServer > 0 EXEC sp_OAMethod  @SQLServer, 'Disconnect'
        IF @SQLServer > 0 EXEC sp_OADestroy @SQLServer
GO

USP_KillConnections

This script (in T-SQL code) can be used or killing the connections to the database. It takes in a parameter, the name of the database, for which the connections have to be killed and also sends out a notification to the connected users that their session is going to be terminated. It uses the Net Send command of Windows to do that.

IF EXISTS (SELECT * FROM SYSOBJECTS WHERE ID = OBJECT_ID('DBO.USP_KillConnections')
AND SYSSTAT & 0XF = 4)
DROP PROCEDURE DBO.USP_KillConnections
GO

CREATE PROCEDURE  USP_KillConnections @USRDB varchar(30) AS

-- DECLARE VARIABLES

declare @spid smallint,
@status varchar(32),
@loginame varchar(32),
@hostname varchar(32),
@blk char(8),
@dbname varchar(32),
@cmd varchar(255),
@KILLSTATE varchar(32),
@USRCOUNT varchar(3),
@PRINTUSRCOUNT VARCHAR (255),
@NOTIFYSTATE varchar (255),
@NOTIFYMSG1 VARCHAR (255),
@NOTIFYMSG2 VARCHAR (255),
@NOTIFYMSG3 VARCHAR (255)

-- CREATE TEMPORARY TABLE FOR SP_WHO TO POPULATE
create table #usrkill (
spid smallint,
ecid smallint,
status varchar(32),
loginame varchar(32),
hostname varchar(32),
blk char(8),
dbname varchar(32),
cmd varchar(255))

-- INSERT INTO TEMP TABLE
insert into #usrkill exec sp_who

-- END PRE-CODE

-- PRINT USER COUNT IN DATABASE

SELECT @USRCOUNT = (SELECT CONVERT(CHAR, COUNT(*)) FROM #usrkill WHERE DBNAME =
@USRDB)

-- IF THERE ARE NO USERS, RAISE EXCEPTION AND NOT RUN KILL STATEMENT

IF @USRCOUNT != '0'
BEGIN
SELECT @PRINTUSRCOUNT = 'Attempting to disconnect ' + @USRCOUNT + 'users'
PRINT @PRINTUSRCOUNT
PRINT 'Attempting to notify users via net send.'

-- BEGIN NOTIFY USERS

-- PREPARING VARIABLES FOR NET SEND
SELECT @NOTIFYMSG1 = '"NET SEND '
SELECT @NOTIFYMSG2 = ' Message from SQL Server : SQL SERVER WILL DISCONNECT YOUR
SESSION IN 30 SECONDS'
SELECT @NOTIFYMSG3 = ""
-- END PREP

DECLARE CNOTIFYUSR SCROLL CURSOR FOR
SELECT DISTINCT HOSTNAME from #usrkill where dbname = @USRDB
OPEN CNOTIFYUSR
FETCH CNOTIFYUSR INTO @hostname

WHILE @@FETCH_STATUS = 0
BEGIN

select @NOTIFYSTATE = @NOTIFYMSG1 + @hostname + @NOTIFYMSG2 + @NOTIFYMSG3
print @notifystate
exec master..xp_cmdshell @NOTIFYSTATE, no_output

FETCH NEXT FROM CNOTIFYUSR into @hostname

END
CLOSE CNOTIFYUSR
DEALLOCATE CNOTIFYUSR

-- END NOTIFY USERS

-- PAUSE SCRIPT FOR 30 SECONDS

WAITFOR DELAY '00:00:29'

-- END PRINT USER COUNT

-- BEGIN CURSOR
DECLARE CUSRKILL SCROLL CURSOR FOR
SELECT SPID from #usrkill where dbname = @USRDB

OPEN Cusrkill
FETCH CUSRKILL into @spid

WHILE @@FETCH_STATUS = 0
BEGIN

select @KILLSTATE = 'KILL ' + CONVERT(char, @spid)
print @KILLSTATE
execute( @KILLSTATE )
FETCH NEXT FROM Cusrkill into @spid
END
CLOSE Cusrkill
DEALLOCATE Cusrkill
END

-- END CURSOR
-- IF NO USERS IN DATABASE RAISE ERROR TO USER
ELSE BEGIN
PRINT 'There are no users in this database to kill.'

END
-- CLEAN UP TEMPDB
DROP TABLE #usrkill
GO

USP_FK_PK

These scripts (in T-SQL code) can be used for getting more information about the primary and the foreign keys.

/*
If you enter the name of the table in place of "x," this piece of T-SQL will give
you the foreign key constraints defined on the table, the name of the child and
parent tables, and the names of the columns involved for the child and the parent
tables.
*/
select      object_name(constid) as constraint_name,
            object_name(fkeyid) as table_name,
            col_name(fkeyid, fkey) as column_name,
            object_name(rkeyid) as referenced_table_name,
            col_name(rkeyid, rkey) as referenced_column_name
from sysforeignkeys
where object_name(fkeyid) = 'x'
order by constraint_name, table_name, referenced_table_name,  keyno

/*
If you replace x with the name of the table you will get the name of the
constraints defined on the table (primary key as well as foreign key constraints),
the names of the columns involved, and the position of the columns in the
constraint, for example, which column is number 1 in the case of a covered primary
key.
*/
select      constraint_name,
            column_name,
            ordinal_position
from information_schema.key_column_usage
where constraint_catalog = db_name()
and table_name = 'x'
order by constraint_name, ordinal_position

USP_DefragDatabase

Over a period of time, because of the DML statements (Insert, Update, and Delete), database fragmentation can take place, which can severely slow down the performance of the database. You can use this script (in T-SQL code) for defragmenting the database.

-- Reindexes the specified database
CREATE PROCEDURE USP_DefragDatabase
-- We don't use sysname because it might not be long enough.
-- sysname is 128 chars, so we use double that.
@dbname nvarchar(256)
AS
BEGIN
    -- Quote the database name with brackets
    DECLARE @quoteddbname nvarchar(256)
    set @quoteddbname = quotename( @dbname )

    -- The outer EXEC permits USE, not allowed in stored procs.
    -- The inner EXEC does the actual reindex on each table in the specified
database.

    EXEC('
    USE '+ @quoteddbname +'
    DECLARE @sTableName sysname
    DECLARE USP_Tables CURSOR LOCAL FOR
        select table_name from information_schema.tables
        where table_type = ''base table'' order by 1
    OPEN USP_Tables
    FETCH NEXT FROM USP_Tables INTO @sTableName

    WHILE @@FETCH_STATUS = 0
    BEGIN
    select @sTablename = quotename(@sTablename, "[]")
        EXEC('' DBCC DBREINDEX ( "+@sTableName+") WITH NO_INFOMSGS")
        FETCH NEXT FROM USP_Tables INTO @sTableName
    END
    CLOSE USP_Tables')
END
GO

USP_DBUpdate

In the development environment as well as the production environment, we often have to make updates to all the databases on the database servers. Under such a scenario, if you have a large number of databases, it is quite cumbersome to run the alter scripts on all the databases every time. By using this script (in T-SQL code), you can update all the databases very easily. This stored procedure takes in two parameters: the directory and the name of the file that contains the SQL script that needs to be run on the databases. If an error occurs during the process of running the Transact-SQL on any database, it will create an error file in the same directory and will name the error file as Database_Name.errorlog.

/*
This script is a stored procedure for performing database updates for all the
databases  on the servers.  The updates need to be performed through the query
analyzer. It takes in two parameters: the directory and the name of the file that
contains the SQL scripts for the update.

 Example: Exec USP_DBUpdate 'C:\temp', 'DBUpdate.sql'
 If an error file is generated, it will be in the same directory and the name will
be the <Databasename>.errorlog.
*/

IF (SELECT COUNT(*) FROM SYSOBJECTS WHERE xtype = 'P' AND NAME = 'USP_DBUpdate') =
1
BEGIN
DROP PROC USP_DBUpdate
END
GO
CREATE PROCEDURE USP_DBUpdate
@dir varchar(256),
@DBUpdatefilename varchar(30)
WITH ENCRYPTION
AS
SET NOCOUNT ON
BEGIN
DECLARE @dbname sysname
DECLARE @file varchar(256)
SELECT @file = @dir + char(92) + @DBUpdatefilename
DECLARE @OutputFile varchar(256)
DECLARE @command varchar(256)

-- Temp table to hold file names for any errors
CREATE TABLE #error
(FileName       varchar(256))

DECLARE Dbcursor_Name cursor for
select name
from master.dbo.sysdatabases
where name not in ('master', 'model', 'pubs', 'tempdb', 'northwind', 'msdb')
and status & 1024 != 1024     -- avoid read only
order by dbid

OPEN Dbcursor_Name

FETCH Dbcursor_Name INTO @dbname

WHILE @@fetch_status = 0
  BEGIN

    -- Quote the database name with brackets
    DECLARE @quoteddbname sysname
    set @quoteddbname = quotename(@dbname)
    print"
    print'Database Name:'print @quoteddbname
    SELECT @OutputFile = @dir + char(92) + @dbname +  '.errorlog'
    Declare @SQLString varchar(2000)

    -- Prepare the SQL String for execution: Set the database name first and then
take the provided filename
    -- and output the errors to the output file

  SELECT @SQLString=  ' Exec(" master..xp_cmdshell '+ """isql -E -
d'+""+'['+@dbname+ ']'+""+' -i'+ @file + ' -o'+"" + @OutputFile + ""+ """ ")'

  EXEC(@SQLString)
    -- Shell out to a command prompt and search the output file for any errors
    select @command = 'master..xp_cmdshell ''findstr /m "Msg"
'+'"'+@OutputFile+'"'+'"'
    print'Check for DBUpdate Errors: Insert into the error table if there are any
errors'
    truncate table #error
    insert into #error exec(@command)

-- Check to see if there were any errors

IF (SELECT COUNT(*) FROM #error) = 1
 BEGIN
    -- No errors found, so file can be deleted
   SELECT @command = 'master..xp_cmdshell ''del ' +""+ @OutputFile + ""+""
   EXEC(@command)
 END
    ELSE
   TRUNCATE TABLE #error

 print 'All updates have been applied successfully TO' print @dbname
 FETCH NEXT FROM Dbcursor_Name INTO @dbname

  END

DEALLOCATE Dbcursor_name
END

SET NOCOUNT OFF
GO

USP_Calc_Space

This procedure (in T-SQL code) will be used for calculating the size of the tables in the database, which helps in estimating the size requirements of the table and the net size of the database.

Note

We can also use sp_spaceused '<name of the object>' to get used space information for that particular object or the database itself. But this procedure gives helpful information about the number of data rows on each page, unused space at the page level, and the number of clustered and nonclustered index pages; it is useful for calculating the size requirements as the data increases. The stored procedure takes in the object name and the number of data records as parameters.

IF exists (SELECT * from sysobjects where id = object_id(N'[dbo].[USP_CalcSpace]')
and OBJECTPROPERTY(id, N'IsProcedure') = 1)
     drop procedure [dbo].[USP_CalcSpace]
GO

CREATE PROCEDURE USP_CalcSpace

--     This script is procedure to estimate the disk space requirements of a table.
--     Refer to Books OnLine topic "Estimating the size of a table..."
--     for a detailed description

      (@table_name     varchar(30) = null,-- name of table to estimate
      @num_rows     int = 0,      -- number of rows in the table
      @var_pct     decimal(3,2) = '1.00')
-- average percent of space used by variable length data types
AS

DECLARE @msg     varchar(120)

--     Give usage statement IF @table_name is null

IF @table_name is null or @num_rows = 0
BEGIN
     PRINT 'Usage is:'
     PRINT '   USP_CalcSpace table_name, no_of_rows, var_pct'
     PRINT '   where table_name is the name of the table,'
     PRINT '   no_of_rows is the number of rows in the table,'
     PRINT '   and var_pct is the decimal percentage of the average'
     PRINT '   variable length column usage.'
     PRINT 'Example: USP_CalcSpace PKT_HDR, 100, 0.5 '
     return
END

DECLARE      @num_fixed_col                int,
     @fixed_data_size                      int,
     @num_variable_col                     int,
     @max_var_size                         int,
     @null_bitmap                          int,
     @variable_data_size                   int,
     @table_id                             int,
     @num_pages                         bigint,
     @table_size_in_bytes              numeric,
     @table_size_in_meg                numeric,
     @table_size_in_kbytes             numeric,
     @sysstat                          tinyint,
     @row_size                             int,
     @rows_per_page                        int,
     @free_rows_per_page                   int,
     @fillfactor                           int,
     @num_fixed_ckey_cols                  int,
     @fixed_ckey_size                      int,
     @num_variable_ckey_cols               int,
     @max_var_ckey_size                    int,
     @cindex_null_bitmap                   int,
     @variable_ckey_size                   int,
     @cindex_row_size                      int,
     @cindex_rows_per_page                 int,
     @data_space_used                   bigint,
     @num_pages_clevel_0                   int,
     @num_pages_clevel_1                   int,
     @num_pages_clevel_x                   int,
     @num_pages_clevel_y                   int,
     @Num_CIndex_Pages                     int,
     @clustered_index_size_in_bytes        int,
     @num_fixed_key_cols                   int,
     @fixed_key_size                       int,
     @num_variable_key_cols                int,
     @max_var_key_size                     int,
     @index_null_bitmap                    int,
     @variable_key_size                    int,
     @nl_index_row_size                    int,
     @nl_index_rows_per_page               int,
     @index_row_size                       int,
     @index_rows_per_page                  int,
     @free_index_rows_per_page             int,
     @num_pages_level_0                    int,
     @num_pages_level_1                    int,
     @num_pages_level_x                    int,
     @num_pages_level_y                    int,
     @num_index_pages                      int,
     @nonclustered_index_size              int,
     @total_num_nonclustered_index_pages   int,
     @free_cindex_rows_per_page            int,
     @tot_pages                         numeric
-- Initialize variables
SELECT     @num_fixed_col                   =0,
     @fixed_data_size                       =0,
     @num_variable_col                      =0,
     @max_var_size                          =0,
     @null_bitmap                           =0,
     @variable_data_size                    =0,
     @table_id                              =0,
     @num_pages                             =0,
     @table_size_in_bytes                   =0,
     @table_size_in_meg                     =0,
     @table_size_in_kbytes                  =0,
     @sysstat                               =0,
     @row_size                              =0,
     @rows_per_page                         =0,
     @num_fixed_ckey_cols                   =0,
     @fixed_ckey_size                       =0,
     @num_variable_ckey_cols                =0,
     @max_var_ckey_size                     =0,
     @cindex_null_bitmap                    =0,
     @variable_ckey_size                    =0,
     @cindex_row_size                       =0,
     @cindex_rows_per_page                  =0,
     @data_space_used                       =0,
     @num_pages_clevel_0                    =0,
     @num_pages_clevel_1                    =0,
     @Num_CIndex_Pages                      =0,
     @clustered_index_size_in_bytes         =0,
     @num_fixed_key_cols                    =0,
     @fixed_key_size                        =0,
     @num_variable_key_cols                 =0,
     @max_var_key_size                      =0,
     @index_null_bitmap                     =0,
     @variable_key_size                     =0,
     @nl_index_row_size                     =0,
     @nl_index_rows_per_page                =0,
     @index_row_size                        =0,
     @index_rows_per_page                   =0,
     @free_index_rows_per_page              =0,
     @num_pages_level_0                     =0,
     @num_pages_level_1                     =0,
     @num_pages_level_x                     =0,
     @num_pages_level_y                     =0,
     @num_index_pages                       =0,
     @nonclustered_index_size               =0,
     @total_num_nonclustered_index_pages    =0,
     @free_cindex_rows_per_page             =0,
     @tot_pages                             =0

SET NOCOUNT ON

-- *********************************************
-- MAKE SURE TABLE EXISTS
-- *********************************************

SELECT  @sysstat = sysstat,
     @table_id = id
     FROM sysobjects WHERE NAME = @table_name

IF @sysstat & 7 not in (1,3)
BEGIN
     SELECT @msg = 'Cannot find the table '+@table_name
     PRINT @msg
     return
END

-- *********************************************
-- ESTIMATE SIZE OF TABLE
-- *********************************************

-- get total number and total size of fixed-length columns

SELECT      @num_fixed_col = count(name),
     @fixed_data_size = sum(length)
     from      syscolumns
     where     id= @table_id and xtype in
               (
               SELECT xtype from systypes where variable=0
               )

     IF @num_fixed_col= 0 -- @fixed_data_size is null.  change to 0
          SELECT @fixed_data_size=0

-- get total number and total maximum size of variable-length columns

SELECT         @num_variable_col=count(name),
     @max_var_size= sum(length)
     from      syscolumns
     where     id= @table_id and xtype in
               (
               SELECT xtype from systypes where variable=1
               )
     IF @num_variable_col= 0 -- @max_var_size is null.  change to 0
          SELECT @max_var_size=0

-- get portion of the row used to manage column nullability

SELECT @null_bitmap=2+((@num_fixed_col+7)/8)

-- determine space needed to store variable-length columns
-- This assumes that all variable length columns will be 100% full,
-- which might not be the case.

IF @num_variable_col = 0
     SELECT @variable_data_size=0
else
     SELECT @variable_data_size =
          2 + (@num_variable_col *2 )+ @max_var_size* @var_pct

-- get row size

SELECT @row_size=      @fixed_data_size +
                       @variable_data_size +
                       @null_bitmap + 4  -- 4 represents the data row header

print 'row_size'
PRINT @row_size

-- get number of rows per page

SELECT @rows_per_page = (8096) / (@row_size+2)

print 'rows per page'
PRINT @rows_per_page

-- If a clustered index is to be created on the table,
-- calculate the number of reserved free rows per page,
-- based on the fill factor specIFied.
-- If no clustered index is to be created, specIFy Fill_Factor as 100.

SELECT      @fillfactor = 100 -- initialize it to the maximum
SELECT      @free_rows_per_page = 0  -- initialize to no free rows/page
SELECT      @fillfactor=OrigFillFactor
from      sysindexes
where      id = @table_id and indid=1  -- indid of 1 means the index is clustered

IF @fillfactor<>0
     -- A 0 fill factor ALMOST fills up the entire page, but not quite.
     -- The doc says that fill factor zero leaves 2 empty rows (keys)
     -- in each index page and no free rows in data pages of clustered
     -- indexes and leaf pages of nonclustered.
     -- We are working on the data pages in this section
     SELECT @free_rows_per_page = 8096 * ((100-@fillfactor)/100)/@row_size

print 'free rows per page'
PRINT @free_rows_per_page

-- Get number of pages needed to store all rows

SELECT @num_pages = ceiling(convert(dec,@num_rows) / (@rows_per_page-
@free_rows_per_page))

-- Get storage needed for table data

SELECT @data_space_used = 8192*@num_pages

print 'data space used'
PRINT @data_space_used

-- *********************************************
-- COMPUTE SIZE OF CLUSTERED INDEX IF ONE EXISTS
-- *********************************************

-- Create a temporary table to contain columns in clustered index.
-- System table sysindexkeys has a list of the column numbers contained in the
index

SELECT colid into #col_list
from sysindexkeys where id= @table_id and indid=1  -- indid=1 means clustered

IF (SELECT count(*) from #col_list) >0 -- do the following only IF clustered index
exists
BEGIN
     -- Get total number and total maximum size of fixed-length columns in clustered
index

SELECT      @num_fixed_ckey_cols=count(name),
          @fixed_ckey_size= sum(length)
          from      syscolumns
          where      id= @table_id and xtype in
          (
          SELECT xtype from systypes where variable=0
          )
          and colid in (SELECT * from #col_list)

     IF @num_fixed_ckey_cols= 0 -- @fixed_ckey_size is null.  change to 0
          SELECT @fixed_ckey_size=0

          -- Get total number and total maximum size of variable-length columns in
          -- clustered index

          SELECT     @num_variable_ckey_cols=count(name),
          @max_var_ckey_size= sum(length)
          from      syscolumns
          where     id= @table_id and xtype in
          (
          SELECT xtype from systypes where variable=1
          )
          and colid in (SELECT * from #col_list)

     IF @num_variable_ckey_cols= 0 -- @max_var_ckey_size is null.  change to 0
          SELECT @max_var_ckey_size=0

     -- If there are fixed-length columns in the clustered index,
     -- a portion of the index row is reserved for the null bitmap. Calculate its
size:
     IF @num_fixed_ckey_cols <> 0
          SELECT @cindex_null_bitmap=2+((@num_fixed_ckey_cols + 7)/8)
     else
          SELECT @cindex_null_bitmap=0

     -- If there are variable-length columns in the index, determine how much
     -- space is used to store the columns within the index row:

     IF @num_variable_ckey_cols <> 0
          SELECT @variable_ckey_size=2+(@num_variable_ckey_cols *2)+@max_var_ckey_size
     else
          SELECT @variable_ckey_size=0

     -- Calculate the index row size

     SELECT @cindex_row_size=@fixed_ckey_size
+@variable_ckey_size+@cindex_null_bitmap+1+8

     -- Next, calculate the number of index rows per page (8096 free bytes per
page):

     SELECT @cindex_rows_per_page=(8096)/(@cindex_row_size+2)

     -- Consider fillfactor
     IF @fillfactor=0
          SELECT @free_cindex_rows_per_page = 2
     else
           SELECT @free_cindex_rows_per_page= 8096 * ((100-@fillfactor)/100)/@cindex_row_size

     -- Calculate the number of pages required to store
     -- all the index rows at each level of the index

     SELECT
@num_pages_clevel_0=ceiling(convert(decimal,(@data_space_used/8192))/(@cindex_rows_
per_page-@free_cindex_rows_per_page))
     SELECT
     @Num_CIndex_Pages=@num_pages_clevel_0
     SELECT @num_pages_clevel_x=@num_pages_clevel_0

     while @num_pages_clevel_x <> 1
     BEGIN
          SELECT
@num_pages_clevel_y=ceiling(convert(decimal,@num_pages_clevel_x)/(@cindex_rows_per_
page-@free_cindex_rows_per_page))
          SELECT @Num_CIndex_Pages=@Num_CIndex_Pages+@num_pages_clevel_y
          SELECT @num_pages_clevel_x=@num_pages_clevel_y
     END
END

-- *********************************************
-- END CLUSTERED INDEX SECTION
-- *********************************************

-- *********************************************
-- BEGIN NONCLUSTERED INDEX SECTION
-- *********************************************

-- Create temp table with nonclustered index info

SELECT sysindexkeys.indid indid , sysindexkeys.colid colid into #col_list2
from sysindexkeys where
sysindexkeys.indid in
(select indid from sysindexes
inner join sysobjects
on sysobjects.id = sysindexes.id
where sysobjects.name = 'authors'
and sysindexes.indid > 1          -- indid=1 means clustered
and sysindexes.name not like '_WA%')     /*have to do this because SQL Server
itself creates temp indexes for internal use which we don't want to use in
calculation
                         since it will give erroneous results*/
and sysindexkeys.id = @table_id

IF (SELECT count(*) from #col_list2) >0 -- do the following only IF nonclustered
indexes exist
BEGIN
     DECLARE @i int  -- a counter variable
     SELECT @i=1 -- initilize to 2, because 1 is id of clustered index

     while @i< 249 -- max number of nonclustered indexes
     BEGIN
          SELECT @i=@i+1 -- look for the next non-clustered index
          -- reinitialize all numbers
          SELECT      @num_fixed_key_cols = 0,
               @fixed_key_size = 0,
               @num_variable_key_cols = 0,
               @max_var_key_size = 0,
               @index_null_bitmap = 0,
               @variable_key_size = 0,
               @nl_index_row_size = 0,
               @nl_index_rows_per_page = 0,
               @index_row_size = 0,
               @index_rows_per_page = 0,
               @free_index_rows_per_page = 0,
               @num_pages_level_0 = 0,
               @num_pages_level_x = 0,
               @num_pages_level_y = 0,
               @Num_Index_Pages = 0

          -- Get total number and total maximum size
          -- of fixed-length columns in nonclustered index
          SELECT      @num_fixed_key_cols=count(name),
               @fixed_key_size= sum(length)
               from      syscolumns
               where      id= @table_id and xtype in
               (
               SELECT xtype from systypes where variable=0
               )
               and colid in (SELECT colid from #col_list2 where indid=@i)
          IF @num_fixed_key_cols= 0 -- @fixed_key_size is null -- change to 0
               SELECT @fixed_key_size=0

          -- Get total number and total maximum size of variable-length columns in
index
          SELECT      @num_variable_key_cols=count(name),
               @max_var_key_size= sum(length)
               from      syscolumns
               where      id= @table_id and xtype in
               (
               SELECT xtype from systypes where variable=1
               )
               and colid in  (SELECT colid from #col_list2 where indid=@i)
          IF @num_variable_key_cols= 0 -- @max_var_key_size is null -- change to 0
               SELECT @max_var_key_size=0

          IF @num_fixed_key_cols = 0 and @num_variable_key_cols = 0 -- there is no
index
               continue
          -- If there are fixed-length columns in the nonclustered index,
          -- a portion of the index row is reserved for the null bitmap. Calculate
its size:
          IF @num_fixed_key_cols <> 0
               SELECT @index_null_bitmap=2+((@num_fixed_key_cols + 7)/8)
          else
               SELECT @index_null_bitmap=0

          -- If there are variable-length columns in the index, determine how much
          -- space is used to store the columns within the index row:
          IF @num_variable_key_cols <> 0
               SELECT @variable_key_size=2+(@num_variable_key_cols
*2)+@max_var_key_size
          else
               SELECT @variable_key_size=0

          -- Calculate the non-leaf index row size
          SELECT @nl_index_row_size=@fixed_key_size
          +@variable_key_size+@index_null_bitmap+1+8

          -- Calculate the number of nonleaf index rows per page (8096 free
bytes/page):

          SELECT @nl_index_rows_per_page=(8096)/(@nl_index_row_size+2)

          -- Calculate the leaf index row size

          SELECT @index_row_size=@cindex_row_size + @fixed_key_size +
@variable_key_size+@index_null_bitmap+1

          -- Calculate the number of leaf-level index rows per page

          SELECT @index_rows_per_page = 8096/(@index_row_size + 2)

          -- Calcuate the number of reserved free index rows/page based on
fillfactor

          IF @fillfactor=0
          -- a 0 fill factor ALMOST fills up the entire page, but not quite.
          -- The doc says that fill factor zero leaves 2 empty rows (keys)
          -- in each index page and no free rows in data pages of clustered
          -- indexes and leaf pages of nonclustered.
          -- We are working on the nonclustered index pages in this section
               SELECT @free_index_rows_per_page=0
          else
               SELECT @free_index_rows_per_page= 8096 * ((100-
@fillfactor)/100)/@index_row_size

          -- Calculate the number of pages required to store
          -- all the index rows at each level of the index
          SELECT @num_pages_level_0=ceiling(convert(decimal,@num_rows)/@ind
          SELECT
@num_pages_level_0=ceiling(convert(decimal,@num_rows)/@index_rows_per_page-
@free_index_rows_per_page)

          SELECT @Num_Index_Pages=@num_pages_level_0
          SELECT @num_pages_level_x=@num_pages_level_0

          while @num_pages_level_x <> 1
          BEGIN
               SELECT @num_pages_level_y

=ceiling(convert(decimal,@num_pages_level_x)/@nl_index_rows_per_page)
               SELECT
               @Num_Index_Pages=@Num_Index_Pages+@num_pages_level_y
               SELECT @num_pages_level_x=@num_pages_level_y
          END

          SELECT @total_num_nonclustered_index_pages
           =@total_num_nonclustered_index_pages+@Num_Index_Pages
     END
END
-- *********************************************
-- END NONCLUSTERED INDEX SECTION
-- *********************************************
-- display numbers

/*
Calculate the final numbers for the computations of the different parameters.
*/

SELECT @tot_pages=@num_pages + @Num_CIndex_Pages +
@total_num_nonclustered_index_pages
SELECT @table_size_in_bytes = 8192*@tot_pages
SELECT @table_size_in_kbytes = @table_size_in_bytes/1024.0
SELECT @table_size_in_meg = str(@table_size_in_kbytes/1000.0,17,2)

print 'total pages'
PRINT @tot_pages
print 'table size in bytes'
PRINT @table_size_in_bytes
print 'table size in kB'
PRINT @table_size_in_kbytes
print 'table size in MB'
PRINT @table_size_in_meg

SELECT      substring(@table_name,1,20) as 'Table Name',
     convert(varchar(10),@table_size_in_meg) as 'MB Estimate',
     @tot_pages as 'Total Pages',
     @num_pages as '#Data Pgs',
     @Num_CIndex_Pages as '#Clustered Idx Pgs',
     @total_num_nonclustered_index_pages as '#NonClustered Idx Pgs'

BackUp.vbs

A very simple VBScript file uses SQL-DMO to specify the backups.

Dim oServer
Dim oDatabase
Dim oBackup
Dim sBAKFilePath

'change this to wherever you want to place your backup files, no trailing
'backslash, we add it below

sBAKFilePath = "C:\BackUp"

'we need a backup object in addition to the sqlserver one

Set oServer = CreateObject("SQLDmo.SqlServer")
Set oBackup = CreateObject("SQLDmo.Backup")
oServer.LoginSecure = True
oServer.Connect "(local)"

'this will do a full backup of every database except TempDB to a file

For Each oDatabase In oServer.Databases
    If UCase(oDatabase.Name) <> "TEMPDB" Then
        oBackup.Database = oDatabase.Name
        'remove any previous backup - same as using T-SQL with init
        oBackup.Initialize = True
        'dynamically create the name of the backup file
        oBackup.Files = sBAKFilePath & "\" & oDatabase.Name & ".bak"
        'set the action property as needed
        '0 = Full backup
        '1 = Differential
        '2 = Specified files only
        '3 = Log backup
        oBackup.Action = 0
        oBackup.SQLBackup oServer
    End If
Next

'clean up
Set oBackup = Nothing
oServer.DisConnect
Set oServer = Nothing
MsgBox "Database BackUps Done"

USP_PingServers

This script (in T-SQL code) can be used to check whether the database servers are up and running. It will print the detailed report as an html page to the specified location.

/*
** 1- Should be created in master database
** 2- SQL Mail has to be running on the sql server.
*/

Create Procedure USP_PingServers as
begin
declare @servname varchar(20)
declare @subject2 varchar(30)
declare @message2  varchar(500)
declare @command  varchar(1000)
    if (select count(*) from sysobjects where xtype = 'u' and name =
'server_names') = 0
     begin
       create table  server_names
               (server_output  varchar(8000))
     end

declare server_name cursor for select srvname from machines order by srvname
open server_name
fetch next  from server_name into @servname
WHILE @@FETCH_STATUS = 0
 BEGIN
select @command = 'master..xp_cmdshell "ping '+"+@servname+"+""
       Insert into server_names EXEC (@command)
         select * from  server_names where server_output like '%timed out%'
  if @@rowcount>0
     Begin
      select @subject2= @servname + ' is down '
      select @message2 = 'You will receive an e-mail when it is back up again'
       if @servname='microndt2978-2'
          EXEC master..xp_sendmail @recipients = 'RASHARMA',
               @subject =@subject2,
               @message = @message2
       if @servname='ma-atl60'
          EXEC master..xp_sendmail @recipients = 'RASHARMA',
               @subject =@subject2,
               @message =@message2
       if @servname='ma-atl25'
          begin
          EXEC master..xp_sendmail @recipients = 'RASHARMA',
               @subject =@subject2,
               @message =@message2

          update machine_status set status = 'DOWN' where machine_name = 'ma-atl25'
          exec sp_makewebtask @outputfile = N'\\microndt2978-
2\d$\inetpub\wwwroot\test.htm', @query='select * from machine_status'
          end
         end -- @@rowcount
      Fetch next from  server_name into @servname
      truncate table server_names
 End -- While
close server_name
DEALLOCATE server_name
drop table server_names
end
go

Schema Comparison Scripts

These scripts can be divided into three categories:

  • Schema comparison between Oracle and SQL Server databases after conversion from one to the other

  • Schema comparison between two SQL Server databases

  • Schema comparison between two Oracle databases

These stored procedures (in T-SQL code) use the concept of linked servers. First a linked server definition needs to be created from SQL Server to Oracle database(s) and SQL Server database(s). Then either of these procedures can be executed to compare the schemas to make sure that they are in synch. (See Chapter 21 for how to create linked server definitions.)

The T-SQL code for these procedures is very long and is included on the CD-ROM under these SQL files:

  • USP_CompareOra2Ora: For comparing two Oracle schemas.

  • USP_CompareSS2SS: For comparing two SQL Server database schemas.

  • USP_CompareOra2SS: For comparing Oracle and SQL Server schemas after conversion from one to the other.

USP_MonitorServices

This procedure (in T-SQL code) can be used for finding out whether the service on the database server is up and running or not, achieved by using the DumpEl.exe application, which ships in the Microsoft Windows 2000 resource kit and through the use of T-SQL.

Create proc USP_MonitorServices
as
begin
declare @command varchar(400), @count int, @cureventlog varchar(400), @position
int, @machine_name varchar(50), @message varchar(1000),
@subject2 varchar(100), @message2 varchar(100), @time varchar(100)

    if (select count(*) from sysobjects where xtype = 'u' and name =
'tblEvent_Log') = 0
     begin
       create table tblEvent_Log
               (Event_Log  varchar(8000))
     end

truncate table tblEvent_Log
insert into tblEvent_Log exec xp_cmdshell 'DumpEl.exe -s microndt2978-2 -l
application'

select @subject2 = 'Service error occured'
select @time = substring(@cureventlog, 1, 20)
SELECT @Count = Count(*) FROM tblEvent_Log where Event_Log like '%sysmonlog%'
declare cureventlog cursor for select * from tblEvent_Log where Event_Log like
'%sysmonlog%'
open cureventlog
fetch next  from cureventlog into @cureventlog
SET ROWCOUNT 1
WHILE @Count > 0
BEGIN

print @cureventlog
select @position = charindex ('\\', @cureventlog)
select @machine_name = substring(@cureventlog, @position + 2, 14)
print @machine_name
       if @machine_name like 'microndt2978-2%'
          BEGIN
          select @message2 = 'Processor utilization exceeded 80% on '+@machine_name
          EXEC master..xp_sendmail @recipients = 'RASHARMA',
               @subject =@subject2,
               @message = @message2
          END
        fetch next  from cureventlog into @cureventlog
     SET @Count = @Count - 1
END -- Loop
SET ROWCOUNT 0
close cureventlog
DEALLOCATE cureventlog
drop table tblEvent_Log
end
go

Note

Since this procedure checks after fixed periods of time whether the service is up or not, a better way of achieving this would be through the Windows scripting host language rather than through T-SQL.

USP_DTSReplaceOwner

This stored procedure (in T-SQL code) can be used for changing the owner of the DTS packages. It is based on the undocumented stored procedure sp_reassign_dtspackageowner.

if exists (select * from sysobjects where id =
object_id(N'[dbo].[USP_DTSReplaceOwner]') and OBJECTPROPERTY(id, N'IsProcedure') =
1)
drop procedure [dbo].[USP_DTSReplaceOwner]
go
CREATE PROCEDURE USP_DTSReplaceOwner
@old_owner sysname,
@new_owner sysname
AS

 -- USP_DTSReplaceOwner
 -- All packages owned by @old_owner are updated with @new_owner
 -- Uses undocumented MS sp, sp_reassign_dtspackageowner

 DECLARE @name sysname, @id uniqueidentifier

 IF (NOT EXISTS (SELECT * FROM sysdtspackages WHERE [owner] = @old_owner))
 BEGIN
     DECLARE @errMsg varchar(255)
     SET @errMsg = 'Owner ' + @old_owner + ' does not own any packages'
     RAISERROR(@errMsg, 16, 1)
     RETURN
 END

 DECLARE cur_sysdtspackages CURSOR FOR
   SELECT DISTINCT [name], [id]
   FROM sysdtspackages
   WHERE [owner] = @old_owner
 OPEN cur_sysdtspackages

 FETCH NEXT FROM cur_sysdtspackages
 INTO @name, @id

 WHILE @@FETCH_STATUS = 0
 BEGIN
     EXEC sp_reassign_dtspackageowner @name=@name, @id=@id,
@newloginname=@new_owner

     FETCH NEXT FROM cur_sysdtspackages
     INTO @name, @id
 END
GO


DTS Packages

You can program DTS packages to set up certain administration tasks (Figure 27.1).

Figure 27.1. DTS Package for refreshing databases on a server

Refresh Databases

The package asks for three parameters in the "Get Global Variables" step as input parameters:

  • Name of the server

  • Name of the source database

  • Name of the destination database

These three parameters are set in the second step, "Set Parameters," which sets the dynamic parameters for the subsequent steps. The third step, "Kill Connections," kills the connections to the destination database to prevent any session from holding locks. The next step "Disable Constraints" disables the constraints in the destination database so that the data can be copied over in a sequential manner without heading to the constraints in the database. The next step "Copy Data" is the step that does the work of copying the database objects and the data from the Source database to the Destination database. The next step "Enable Constraints" enables the constraints in the destination database. The entire DTS Package utilizes two SQL Server connections: one to the master database and the second one to the destination database, the name of the database is set at run-time after the user enters the name of the database. The scripts used in this DTS package have already been discussed in this chapter. This DTS package makes use of the USP_KillConnections, USP_Disable_ALL and USP_Enable_ALL T-SQL scrips.

Back Up Databases and Version Control

In this script, the first step, "Back Up Database(s)," executes a backup Transact-SQL statement and creates backups of the databases on the disk drive. The next step, "Copy Backup Files," copies the backups to a directory structure from whence they are checked into a version control software like Microsoft Visual SourceSafe or Star-Team. The third step, "Delete the Backup Copy," deletes the backups from the disk drive to reclaim space. This step can be used for scheduling backups of the development server databases (see Figure 27.2). For production servers, the backups should ideally be taken to the tape drive itself.

Figure 27.2. Back up databases and version control

ScriptData

This is a sample application written in VB that you can use to generate the insert statements for the tables in the database, achievable either through the use of stored procedures or by utilizing SQL-DMO to connect to the tables and then script the insert statements. The entire code is contained under the ScriptData folder.

ExportPackages.vbs

This VBScript can be used for exporting the DTS packages to a folder for archiving.

option explicit

'-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -
' ExportPackages.vbs -- exports DTS packages to .dts files.
'
' Usage:  cscript ExportPackage.vbs <package folder> <SQL Server name> <SQL
user> <SQL password>
'
' Example:  cscript ExportPackage.vbs "..\setup\VB Scripts\DTS Packages"
microndt2978-2 sa password
'
' This script exports all packages on a SQL Server to the specified folder,
' one package per file.
'
' NOTE: THE CONTENTS OF THE <package folder> FOLDER ARE ERASED.
'-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -

dim sPath
dim sPackageFolder
dim sServer
dim sUser
dim sPassword
dim iCount

dim oFolder
dim oPackage

' Validate arguments.
if wscript.arguments.count < 4 then
    wscript.echo "Usage:  ExportPackages <package folder> <SQL server name>
<SQL user> <SQL password>"
    wscript.quit 1
end if

sPackageFolder = wscript.arguments(0)
sServer = wscript.arguments(1)
sUser = wscript.arguments(2)
sPassword = wscript.arguments(3)

' Create objects.
dim fso
set fso = CreateObject("Scripting.FileSystemObject")

dim oConn
set oConn = CreateObject("ADODB.Connection")

dim rs
set rs = CreateObject("ADODB.Recordset")

' Delete the specified folder.
on error resume next
fso.DeleteFolder sPackageFolder
on error goto 0

' Recreate the folder.
set oFolder = fso.CreateFolder(sPackageFolder)
' Connect to the specified SQL Server using the specified user id and
password.
oConn.ConnectionString = _
    "Data Source=" & sServer & ";" & _
    "Initial Catalog=msdb;" & _
    "Provider=SQLOLEDB.1;" & _
    "Persist Security Info=False;" & _
    "User Id=" & sUser & ";" & _
    "Password=" & sPassword & ";"
oConn.Open

wscript.echo "Connected to " & sServer

' Get a list of packages.
rs.Open "exec msdb..sp_enum_dtspackages", oConn

' Tell the user if no packages were found.
if rs.EOF then
    wscript.echo "No DTS packages found.  Verify with SQL Enterprise
Manager."
    wscript.quit 1
end if

iCount = 0

' Iterate through the package list.
do until rs.EOF
    ' Load the package from SQL Server.
    set oPackage = CreateObject("DTS.Package2")
    oPackage.LoadFromSQLServer sServer, sUser, sPassword, , , , , rs("name")

    wscript.echo "Exporting " & rs("name") & "..."

    ' Save the package to a file.
    oPackage.SaveToStorageFile sPackageFolder & "\" & rs("name")

    iCount = iCount + 1

    ' Release the package object.  It can't be reused.
    set oPackage = Nothing

    ' Point to the next package.
    rs.MoveNext
loop
wscript.echo

if iCount = 1 then
    wscript.echo "1 package exported successfully."
else
    wscript.echo iCount & " packages exported successfully."
end if
wscript.quit 0

ImportPackages.vbs

This VBScript can be used for importing the DTS packages from a folder.

option explicit

'-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -
' ImportPackages.vbs -- import DTS packages from .dts files.
'
' Usage:  cscript ImportPackages.vbs <sql server name> <sql server userid>
<sql server password>
'
' Example:  cscript ImportPackages.vbs microndt2978-2 sa password
'
' This script imports all packages from the folder specified below to the
' specified SQL Server.
'
'-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -

const PACKAGE_FOLDER = "DTS Packages"

dim oPackage
dim oFile
dim sPath
dim iCount
dim sPackageName
dim sServer
dim sUserid
dim sPassword
dim oArgs

' Validate arguments.
set oArgs = wscript.Arguments
if oArgs.count < 3 then
    wscript.echo "Usage:  cscript ImportPackages.vbs <sql server name> <sql
server userid> <sql server password>"
    wscript.quit 1
end if

sServer = oArgs(0)
sUserid = oArgs(1)
sPassword = oArgs(2)

' Obtain the path that the script is run in.  The DTS package folder
' is relative to that path.
sPath = left(wscript.scriptfullname, len(wscript.scriptfullname) -
len(wscript.scriptname))

dim fso
set fso = CreateObject("Scripting.FileSystemObject")

dim oFolder

' Make sure the package folder exists.
if not fso.FolderExists(sPath & PACKAGE_FOLDER) then
    wscript.echo "Can't find package folder " & sPath & PACKAGE_FOLDER
    wscript.quit 1
end if

set oFolder = fso.GetFolder(sPath & PACKAGE_FOLDER)

iCount = 0

' Iterate through the package files.
for each oFile in oFolder.Files
    wscript.echo "Importing " &  oFile.Path
    set oPackage = CreateObject("DTS.Package2")

    ' Get the package name, which is the name of the .dts file
    ' without the .dts extension.
    sPackageName = left(oFile.Name, len(oFile.Name) - 4)

    ' Delete the package, but don't error out if it already doesn't exist.
    on error resume next
    oPackage.RemoveFromSQLServer sServer, sUserid, sPassword, , , ,
sPackageName
    on error goto 0

    ' Load the package from the .dts file.
    oPackage.LoadFromStorageFile oFile.Path, ""

    ' Save the package to SQL Server.
    oPackage.SaveToSQLServer sServer, sUserid, sPassword

    iCount = iCount + 1

    ' Release the package object.  It can't be reused.
    set oPackage = Nothing
next

wscript.echo "Successfully imported " & iCount & " packages."
wscript.quit 0

Executing DTS Through T-SQL

This procedure (in T-SQL code) can be used for executing the DTS packages through stored procedures:

CREATE procedure USP_dtsexecute (@srv varchar(10),@login varchar(20), @pwd
varchar(20),@pkg varchar(25))
as
declare @str varchar(100)
declare @ret int
select @str="dtsrun /S" +ltrim(@srv) + " /U" + ltrim(@login) + " /P" +ltrim(@pwd) +
" /N" + ltrim(@pkg)
exec @ret = xp_cmdshell @str, no_output
return @ret


Copying Databases from Server to Server

Databases can be copied a number of ways.

Method 1

The fastest way to copy a database is by detaching the database from source server and then attaching the data and the log files to the destination server.

T-SQL

Detach the database on the source server (Place the name of the database.):

Use Master
GO
Exec sp_detach_db 'database_name', 'true'
GO

Sp_detach_db detaches the database from the source server (there are two parameters: @dbname, which is the name of the database, and @skipchecks, which is for updating statistics), and a value of true for the second parameter (@skipchecks) of this stored procedure makes sure that Update Statistics is not run before detaching the database from the server (updating statistics takes some time).

Copy over the the data and the log files from the data directory of the source server to the data directory on destination server. (Make sure the database is not already on the destination server. If it is, drop it first before copying the files).

Attach the data and the log files to your destination server. Execute the following on the destination server:

Use Master
GO
PRINT 'Attaching Database'
EXEC sp_attach_db @dbname = 'database_name',
@filename1 = 'c:\mssql7\data\database_name.mdf',    -- This is the path to your data
file
@filename2 = 'c:\mssql7\data\database_name_log.ldf' -- This is the path to your log
file

This code attaches the database to the destination server, but it does not copy over the logins from server 1 to server 2. Use the following script to do so.

/* Setup a linked server (using sp_addlinkedserver and sp_addlinkedsrvlogin) called
sourceserver from which the standard logins needs to be transferred. You can call
it whatever you want and modify the linked server name also. You also need to use
sp_serveroption to enable data access for the linked server.*/
declare @login sysname , @password sysname
declare sourcelogins cursor for
  select name , password
  from sourceserver.master.dbo.syslogins
  where isntname = 0 and charindex( 'repl_' , name ) = 0 and
        charindex( 'distributor' , name ) = 0 and name != 'sa'
open sourcelogins
while ( @@fetch_status = 0)
begin
 fetch sourcelogins into @login , @password
 exec sp_addlogin @login , @password , @encryptopt = 'skip_encryption'
end
close sourcelogins
deallocate sourcelogins
go

When detaching and attaching databases, I have run into another problem in addition to the orphaned users described earlier. The statistics can be out of date and poor performance can occur unless sp_updatestats is run on the newly attached database. I therefore recommend updating statistics after you have attached the database. As an alternative, when you detach the database, make sure that the second parameter is set to false. That will run updatestats before detaching the database. That way you don't need to run it later on the destination server.

This first copying method also works well if you have to transfer the databases to a disk and then attach them to a destination server that is not on the network. It's also very fast way of copying databases from server to server. Do remember to reattach it to the source server once you have copied over the data and the log files from the source server to the destination server.

Method 2

DTS works well for moving databases from server to server. You can use the DTS Import Export Wizard (SQL 7.0 and SQL 2000), which can be used to copy over the schema, the objects (stored procedures, views and triggers, and so on), data and the logins as well, or you can use the DTS designer and create a transfer database task and transfer logins task (available only in SQL Server 2000). You can also use the Copy Database Wizard (only in SQL Server 2000). DTS is a pretty powerful tool, and if you have not used it already, you are missing out on the functionality and the ease with which you achieve complex results.

The Refresh_Database.DTS package we talked about earlier can be used for copying databases. It can easily be extended to copy databases from one server to another server by including another global variable for the destination server name.

Method 3

Create the schema on the destination server if you already have the script for schema creation and then use bcp or bulk insert to pump in the data. Both are nonlogged operations, so they are pretty fast. The major difference between bcp and BULK INSERT is that BULK INSERT cannot export data whereas BCP can.

You can use BCP to export data to a flat file and then import data into the database from the flat file using either bcp or BULK INSERT. Even though BULK INSERT is a fast way to pump in data, it has a big limitation and that is that the source data must align perfectly to the destination table.

Method 4

Make a backup copy of the database and then restore it on the destination server.

Method 5

You first need to create the schema at the destination server by using your schema creation scripts. Then you can create a linked server and write insert statements to insert data from the source to the destination server using the Openrowset and Openquery functions for querying linked servers. You must disable the foreign key and check constraints before you start pumping in data and then reenable them when your done. This method is the slowest of the five.


Summary

These scripts should help in easing some of the database administration and maintenance tasks for DBAs and the database developers.


Part V: SQL Server.NET

What is Microsoft.NET and why should I be concerned about it? These were the two questions that came to my mind when a colleague of mine mentioned .NET to me a couple of months back. How does this affect the world of the DBAs was a question that another SQL Server DBA in our organization asked. In this part we try to understand what Microsoft.NET really means and how SQL Server 2000 and the next release of SQL Server fit into the .NET framework.


Chapter 28. SQL Server 2000 and .NET

Although Microsoft only recently completed SQL Server 2000, the company has been working on its follow-up, which will take SQL Server fully into the Web-enabled .NET environment. The next release of SQL Server which probably will ship in 2003, will extend SQL Server's XML capabilities and add several ease-of-use features targeted at administrators and developers.


What Is Microsoft.NET?

At first I thought that Microsoft.NET was just another buzzword for the next generation of tools and servers and that as a database professional I should not even be concerned about .NET. I'm not a developer. Sure, I can sling Visual Basic (VB) code if I have to, but that isn't how I earn my living. I need to know how Visual Studio applications will affect my database, so I keep up with ADO and middleware technology. But I don't get excited when Microsoft starts talking about a new set of programming tools. I thought, "Great, Microsoft improved VB and added threading." But when I learned more about .NET, I realized how wrong I was. The .NET Framework could fundamentally change the way we design, build, and deploy applications. .NET Enterprise Servers—including SQL Server 2000—are cool. The new .NET programming tools and common runtime are nifty.


From Products Vendor to Services Vendor

One of .NET's key goals is to transform Microsoft from a products vendor into a services vendor. As part of the .NET initiative, Microsoft will morph many of its products into services that you can subscribe to and use across the Internet. For instance, Microsoft has stated its plans to deliver two versions of the next release of Office: (1) a subscription-based service that users access across the Internet and (2) a traditional shrink-wrapped version of Office.NET that you install on your local system. Microsoft will continue to deliver other products, such as SQL Server and other members of the BackOffice family, as stand-alone products. Microsoft has yet to provide details about the pricing models it will use with subscription-based licensing. This transformation into an Internet services company will let Microsoft provide better customer service and make software easier to deploy and install. Microsoft also intends this model to facilitate customer upgrades to the latest software enhancements.

Microsoft.NET is an add-on runtime environment that runs on the Windows 2000 and Windows XP operating systems. It makes it easier for developers to write good, robust code quickly and to manage, deploy, and revise the code. The programs and the components you write execute inside this environment. It provides developers with runtime features such as automatic memory management (garbage collection) and easy access to all system services. It also adds many utility features like easy Internet and database access. The .NET framework is easy to deploy because it does not require registry settings. All of these features are available in any .NET-compliant language.

.NET introduces a new programming model for constructing HTML pages, named ASP.NET. Most Internet traffic uses a generic browser as the front end. This requires a server that constructs a page using HTML that browsers understand and can display to the user. ASP.NET(the next version of Active Server Pages) is a new environment that runs on the Microsoft Web server IIS (Internet Information Services) and makes it easier for the developers to write code that constructs HTML-based Web pages for browser viewing. AS.NET features a new language-independent way of writing code and tying it to Web page requests. It features .NET Web Forms, which is an event-driven programming model of interacting with controls that makes programming a Web page feel very much like programming a Visual Basic form. ASP.NET contains good session state management and security features, and it is more robust and contains many performance enhancements over the original ASP.

Microsoft.NET provides a new way for Internet Servers to expose functions, to any client named .NET Web Services. I think that in the future the Web will become more of a place where, instead of data being rendered in a browser, a dedicated client (say music.com or Napster) will make cross-Internet function calls to a server and receive data to be displayed in a user interface or maybe even without a user interface at all for machine-to-machine communications. .NET provides a new set of services that allows a server to expose its functions to any client on any machine running any operating system. The client makes calls to the server using XML and HTTP. A set of functions exposed in this manner is called a .NET Web Service.

Another example is that of the TerraServer. Remember TerraServer? Microsoft launched it a few years ago as the world's largest SQL Server Application. If you ever need a satellite picture of an area, terraserver.microsoft.com is the place to go. Microsoft has upgraded TerraServer and relaunched it as a Web service called the TerraService. Today, the U.S. Department of Agriculture (USDA) has integrated the TerraService into an application available to farmers and other professionals in the agriculture business. Conceptually, USDA developers can invoke methods and properties from TerraService as if it were a simple COM component installed on the local network. Using a pocket PC, a farmer in the field can receive topographical or aerial photographs of any farm in the United States. Microsoft and TerraServer alone could never have met the unique needs of this agricultural application, and the USDA could never have built its own TerraServer without leveraging the capabilities of a programmable Web service.

Together, combined applications are greater than the sum of their parts. Web services transform the Web from a browseable resource into a programmable resource, bringing us into an entirely new realm of architectural options. Here's what the TerraServer home page has to say about the new Web service:

The Microsoft TerraServer .NET Web Service is designed to simplify the task of accessing TerraServer programmatically. Developers call TerraServer Web Service methods to perform standard queries against the TerraServer database. The Microsoft .NET technologies do the heavy lifting to connect programs to the TerraServer Web Service via Internet communication protocols and exchange data. All the programmer has to do is call the provided TerraServer Web Service API methods and iterate through the data contained in the data structures returned by TerraServer Web Service.

Web services make the dream of true, loosely coupled, distributed-application architectures a reality. But what does this mean to database architects like you and me? Building and distributing Web applications will soon be a mass-market skill. But making these distributed applications actually work will still be difficult, especially when these distributed, loosely coupled Web services share data that might reside down the hall or halfway around the world.

A dedicated client application that uses Web Services needs to provide a good user interface. Microsoft.NET provides a new package, called .NET Windows Forms, that makes it easier to write dedicated Windows client applications using the .NET framework.

Most Internet programs gather information from a client by making a database query and presenting the results to the client. .NET provides good support for database operations using ADO.NET. ADO.NET model allows for multiple tables being stored/passed in data sets.


Web Services

Like COM components, Web Services are black boxes: You don't have to know how they're implemented to use them. Also like COM components, Web Services have well-defined interfaces. The interface for a Web Service, which Microsoft calls a contract, describes the messages that the Web server accepts and generates. In a marked departure from Windows DNA, Microsoft's COM-based n-tier architecture, applications built within the Microsoft .NET Framework don't use Microsoft's proprietary Distributed COM (DCOM) protocol to interact with Web Services.

Most of the .NET building-block services will be available both online and offline. You'll be able to invoke a service on a stand-alone desktop system, on a local service, or across the Internet. You'll be able to invoke .NET building-block services on any platform that supports XML. However, Windows.NET will offer the best environment for creating and delivering Web Services, and Microsoft will optimize Windows-based clients to distribute Web Services to every kind of device.

Much like Java's Java Virtual Machine (JVM), the Microsoft .NET common language runtime (CLR) runs on top of the basic OS services. Also like Java, the common language runtime doesn't execute machine-specific code. Instead the common language runtime, which is a runtime interpreter, executes a source-code-independent language called Microsoft intermediate language (MSIL). MSIL provides a hardware-abstraction layer that insulates the application code from the underlying hardware platform; theoretically, you could implement an MSIL runtime on a non-Windows platform.


.NET Servers

The .NET Enterprise Servers, including the Windows 2000 Server family, make up the Microsoft .NET server infrastructure for deploying, managing, and orchestrating XML Web services. Designed with mission-critical performance in mind, they provide businesses with the agility they need to integrate their systems, applications, and partners through XML Web services and the flexibility to adapt to changing business requirements. Following are the .NET Enterprise Servers:

  • Application Center 2000 to deploy and manage highly available and scalable Web applications

  • BizTalk Server 2000 to build XML-based business processes across applications and organizations

  • Commerce Server 2000 for quickly building scalable e-commerce solutions

  • Content Management Server 2001 to manage content for dynamic e-business Web sites

  • Exchange Server 2000 to enable messaging and collaboration, anytime, anywhere

  • Host Integration Server 2000 for bridging to data and applications on legacy systems

  • Internet Security and Acceleration Server 2000 for secure, fast Internet connectivity

  • Mobile Information 2001 Server to enable application support by mobile devices like cell phones

  • SharePoint Portal Server 2001 to find, share, and publish business information

  • SQL Server 2000 to store, retrieve, and analyze structured XML data


SQL Server 2000 and .NET

Ah! The topic we've been waiting to discuss. The Microsoft Servers Web site (http://www.microsoft.net/servers) lists SQL Server 2000 as one of the company's new .NET Enterprise Servers. Microsoft did add support for XML Updategrams and XML Bulk Load in January 2001 with its XML for SQL Server 2000 Web Release 1 (WR1), which you can download along with the even more recent Web Release 2 (WR2) beta 1 from http://www.microsoft.com/sql/downloads/default.asp. But XML for SQL Server 2000 WR1's enhanced XML integration features didn't boost SQL Server into .NET land. So, is SQL Server 2000's inclusion as a .NET server more a marketing strategy than an architecture enhancement? Yes—and no.

Although .NET came to light after the release of SQL Server 2000, Microsoft was working on .NET while SQL Server 2000 was in beta. The SQL Server development team kept its focus on fulfilling an SQL Server design specification already in place. Still, SQL Server developers were able to implement parts of the fledgling .NET components into the final release of SQL Server 2000.

SQL Server is now XML-enabled and is the foundation server product on which to build .NET XML Web Services. With .NET, Microsoft has opened up the platform to a whole set of services and devices by embracing XML standards that the industry has defined. This strategy differs from their strategies in the past in that so much application development with .NET is based on interoperability with other devices and other services in a clear, documented, standard way. Web Services provides a well-defined way to develop, deploy, and manage applications on the Web and to open up these services to other applications. By combining all these Web services, complex applications can be built by using very simple, industry-standard concepts.

Is SQL Server 2000, with its partial support for some .NET components, a true .NET Server? As I see it, three main attributes define a .NET Server:

  1. A .NET Server requires deep Web and XML integration.

  2. A .NET Server supports Web Services access through SOAP (simple object access protocol). SOAP is another protocol that extends HTTP to accommodate remote Web service invocation. The protocol defines a mechanism for passing commands and parameters between HTTP clients and servers, which lets a user request action on port 80 from a Web service on another Web server across the world. Any programming language on any OS can be used.

  3. A .NET Server provides integration with the .NET Framework and the Common Language Runtime (CLR).

SQL Server 2000 clearly meets the first requirement of a .NET Server, providing Web and XML integration even apart from XML for SQL Server 2000 WR1. However, SQL Server 2000 doesn't natively support Web Services through SOAP nor does it have any CLR integration except that .NET applications can access SQL Server as a client application. If you examine SQL Server 2000 as it stands today, the database system provides just enough .NET Server attributes to let marketing call it a .NET Server. But just wait. Microsoft plans to include all these attributes in the next release of SQL Server, currently under development.


Next Release of Microsoft Server

Even though Microsoft has not officially released a list of feature sets to be incorporated in the next Microsoft Server release, they have announced a broad overview of what the next release shall incorporate. First, it will be a true .NET server and will include all the attributes for the .NET server mentioned above. Code can be programmed in languages other than Transact-SQL—in fact, basically in any .NET compliant language (C#, Visual C++, VB.NET) . This is good as well as bad from the viewpoint of the businesses—good because of the wider selectivity of the languages to choose from and the advantages of each of them, bad because the database developers and the DBAs will have to know another language(s) for maintaining the code base, a potential maintenance issue.

The next Microsoft server release should be better at self-management: It should have increased ability to self-tune and fix problems automatically. Also, the user will be able to store XML documents in the database and hence will be able to run queries against the database directly, back them up, retrieve them ,and index them.


Windows XP and .NET

Windows XP, which is the new operating system from Microsoft, is the successor to Windows 2000. It is designed to serve as the central information hub for services and act as the smartest of all devices amid a growing constellation of pocket PCs, mobile phones, tablet PCs, digital cameras, and other devices. With native support for XML and SOAP, Windows XP enables a new set of services on the PC and gets your computer ready to take advantage of the .NET Framework and the Windows .NET Server. More information about Windows XP and the different scenarios of the usage of XML Web Services can be obtained at: http://www.microsoft.com/windowsxp/.


Summary

In this chapter we discussed what Microsoft.NET has to offer and how SQL Server 2000 and the next release of SQL Server fit into the .NET framework.


Appendix

    SQL Server Resources

    CD-ROM Materials

    MSDE Install Sample VB Application


SQL Server Resources

http://www.microsoft.com/sql

http://msdn.microsoft.com/sqlserver

http://www.sqlmag.com

http://www.swynk.com

http://www.microsoft.com/downloads

http://www.microsoft.com/data

http://www.sqlservercentral.com


CD-ROM Materials

The CD-Rom contains the code included in the chapters:

Chapters 1–26

SQL-DMO Code Samples

Registered SQL servers

ExecuteImmediate method

Script method

Script to execute a job

Disable triggers

Enable triggers


I l@ve RuBoard Previous Section

MSDE Install Sample VB Application

Chapter 27

T-SQL Code

USP_DisableALL.sql

USP_EnableALL.sql

USP_Find_Lock.sql

USP_Lock_Info.sql

Greatest.sql

Update_Statistics.sql

USP_Active.sql

Objects.sql

Number_Of_Rows.sql

USP_Update_One_Table.sql

USP_System_Info.sql

USP_Null_Columns.sql

USP_Setup_Paths.sql

USP_Identity_Columns.sql

USP_DB_File_Info.sql

USP_NOIndexes.sql

USP_RenameDB.sql

USP_ForceDropDB.sql

USP_Trace_BlackBox.sql

USP_Registry_Info.sql

USP_KillConnections.sql

USP_FK_PK.sql

USP_Defrag_Database.sql

USP_DBUpdate.sql

USP_CaclSpace.sql

USP_PingServers.sql

USP_MonitorServices.sql

USP_CompareOra2Ora.sql

USP_CompareSS2SS.sql

USP_CompareOra2SS.sql

USP_DTSReplaceOwner.sql

USP_DTSExecute.sql

DTS Packages

Refresh_Data

Backup and Version Control

VBScripts

Backup.vbs

ExportPackages.vbs

ImportPackages.vbs

ScriptData VB Package
    I l@ve RuBoard Previous Section