Вы находитесь на странице: 1из 20

One of the most significant characteristics of an export operation is its mode, because the mode largely determines what

is exported. Export provides different modes for unloading different portions of the database. The mode is specified on the command line, using the appropriate parameter. The available modes are as follows:

Full Export Mode Schema Mode Table Mode Tablespace Mode Transportable Tablespace Mode

Full Export Mode A full export is specified using the FULL parameter. In a full database export, the entire database is unloaded. This mode requires that you have the EXP_FULL_DATABASE role. Schema Mode A schema export is specified using the SCHEMAS parameter. This is the default export mode. If you have the EXP_FULL_DATABASE role, then you can specify a list of schemas and optionally include the schema definitions themselves, as well as system privilege grants to those schemas. If you do not have the EXP_FULL_DATABASE role, you can export only your own schema. Table Mode A table export is specified using the TABLES parameter. In table mode, only a specified set of tables, partitions, and their dependent objects are unloaded. You must have the EXP_FULL_DATABASE role to specify tables that are not in your own schema, and only one schema can be specified. Note that type definitions for columns are not exported in table mode. It is expected that the type definitions already exist in the target instance at import time. Also, as in schema exports, cross-schema references are not exported. Tablespace Mode A tablespace export is specified using the TABLESPACES parameter. In tablespace mode, only the tables contained in a specified set of tablespaces are unloaded. If a table is unloaded, its dependent objects are also unloaded. Both object metadata and data are unloaded. In tablespace mode, if any part of a table resides in the specified set, then that table and all of its dependent objects are exported. You must have the EXP_FULL_DATABASE role to use tablespace mode. Transportable Tablespace Mode A transportable tablespace export is specified using the TRANSPORT_TABLESPACES parameter. In transportable tablespace mode, only the metadata for the tables (and their dependent objects) within a specified set of tablespaces are unloaded. This allows the tablespace datafiles to then be copied to another Oracle database and incorporated using transportable tablespace import. This mode requires that you have the EXP_FULL_DATABASE role.

In this sample control file, the numbers that appear to the left would not appear in a real control file. They are keyed in this sample to the explanatory notes in the following list: 1. This is how comments are entered in a control file. See Comments in the Control File. 2. The LOAD DATA statement tells SQL*Loader that this is the beginning of a new data load. See Appendix A for syntax information. 3. The INFILE clause specifies the name of a datafile containing data that you want to load. See Specifying Datafiles. 4. The BADFILE parameter specifies the name of a file into which rejected records are placed. See Specifying the Bad File. 5. The DISCARDFILE parameter specifies the name of a file into which discarded records are placed. See Specifying the Discard File. 6. The APPEND parameter is one of the options you can use when loading data into a table that is not empty. See Loading Data into Nonempty Tables. To load data into a table that is empty, you would use the INSERT parameter. See Loading Data into Empty Tables. 7. The INTO TABLE clause allows you to identify tables, fields, and datatypes. It defines the relationship between records in the datafile and tables in the database. See Specifying Table Names. 8. The WHEN clause specifies one or more field conditions. SQL*Loader decides whether or not to load the data based on these field conditions. See Loading Records Based on a Condition. 9. The TRAILING NULLCOLS clause tells SQL*Loader to treat any relatively positioned columns that are not present in the record as null columns. See Handling Short Records with Missing Data. 10. The remainder of the control file contains the field list, which provides information about column formats in the table being loaded. See Chapter 6 for information about that section of the control file.

Data Pump has three distinct components.

The uniqueness of Data Pump is that all processing is done through the database server and through DIRECTORY objects. To do this there are two internal packages to Oracle and an optional replacement for the original export and import command line utilities (exp & imp). expdp & impdp These to command line utilities are very close to the old standby export & import (exp & imp) utilities. They are not stand-alone utilities in the sense that they use the DBMS_DATAPUMP PL/SQL package to execute the export and import functions. They accept a variety of command line options that, like exp & imp, allow you to pick and choose the objects to be exported and imported. DBMS_DATAPUMP The Data Pump API and can be used independently of expdp & impdp. Is the package accessed to move data and / or metadata between databases. DBMS_METADATA The meta-data API in Oracle and can also be used independently of expdp & impdp. If you remember this is the package we were using in the last two articles for extracting meta-data. I am very interested in how it interfaces with Data Pump.

Advantages and Disadvantages of Datapump


===> Better control on the job running it provides features like start, stop, restart ===> Improved performance because of It is server side technology with parallel streams option ===> Using the parallel streams option, data pump can backup large volume of data quickly ===> Data pump is 15-50% faster than the conventional export import. ===> It has the ability to estimate the job times ===> Failed jobs can be restarted ===> Using exclude/include option we can perform fine-grained object selection ===> Backup jobs can be monitored ===> It has the remapping capabilities ===> It supports the export/import operations over the network. The NETWORK_LINK parameter initiate the export using a database link ===> Using Query parameter DBA can extract the data from tables like SELECT Content parameter gives the flexibility for what to import/export. For example Metadata only, data or both ===> It supports full range of data types ===> It supports cross platform compatibility ===> No need to specify the buffer size like in normal exp/imp ===> It has its own performace tuning features ===> V$session_longops view can be used for the time estimation for the data pump jobs ===> It supports interactive mode that allows the dba to monitor or interact with ongoing jobs ===> Dumps will be in compressed

===> Data can be encrypted ===> XML schemas and XML type is supported by the Data Pump Disadvantages: ===> Export cannot be taken to tape ===> Import will work only with Oracle 10g or above ===> Cannot use with Unix pipes

Data Pump Features

1.Writes either Direct Path unloads External tables (part of cluster, has LOB, etc) 2.Command line interface 3.Writing to external tables 4.DBMS_DATAPUMP Data Pump API 5.DBMS_METADATA Metadata API 6.Checkpoint / Job Restart Job progress recorded in Master Table - All stopped Data Pump jobs can be restarted without loss of data as long as the master table and dump file set remain undisturbed while the job is stopped. It doesnt matter if the job was stopped voluntarily by a user of if the stoppage was involuntary due to a crash,power outage, etc. May be stopped and restarted later Abnormally terminated job is also restartable Current objects can be skipped on restart if problematic

7.Better Job Monitoring and Control Can detach from and attach to running jobs from any location - Multiple clients can attach to a job to see what is going on. Clients may also detach from an executing job without affecting it. Initial job space estimate and overall percent done - At Export time, the approximate size of the job is estimated before it gets underway. The default method for determining this is to estimate the size of a partition by counting the number of blocks currently allocated to it. If tables have been analyzed, statisticscan also be used which should provide a more accurate estimate. The user gets an estimate of how much dump file space will be consumed for the operation. Job state and description - Once the Export begins, the user can get a status of the job by seeing a percentage of how far along he is in the job. He can then extrapolate the time required to get the job completed. Per-worker status showing current object and percent done Enterprise Manager interface available - The jobs can be monitored from any location

8. Interactive Mode for expdp and impdp clients PARALLEL: add or remove workers ADD_FILE: add dump files and wildcard specs STATUS: get detailed per-worker status

STOP_JOB {=IMMEDIATE}: stop the job, leaving it restartable, immediate doesnt wait for workers to finish current work items. START_JOB: Restart a previously stopped job, can change reporting interval KILL_JOB: stop job and delete all its resources, leaving it unrestartable, the master table and dump files are deleted CONTINUE: leave interactive mode, continue logging EXIT: exit client, leave job running

All modes of operation are supported: full, schema, table, tablespace, and transportable tablespace.

9.Dumpfile Set Management Directory based: e.g., DMPDIR:export01.dmp where DMPDIR is external directory Can specify maximum size of each dumpfile

Can dynamically add dumpfiles to jobs - If a job ran out of space, can use ADD_FILE command and specify a FILESIZE value Wildcard file specs supported - Wildcard file support makes it easy to spread the I/O load over multiple spindles: e.g.: Dumpfile=dmp1dir:full1%u.dmp, dmp2dir:full2%u.dmp Dump file compression of metadata - Metadata is compressed before being written to the dumpfile set COMPRESSION=METADATA_ONLY In Oracle Database 11g, this compression capability has been extended so that we can now compress table data on export. Data Pump compression is an inline operation, so the reduced dumpfile size means a significant savings in disk space. Automatically uncompressed during Import. Data Pump compression is fully inline on the import side as well, so there is no need to uncompress a dumpfile before importing it. Dumpfile set coherency automatically maintained Data Pump supplies encryption options for more flexible and robust security.

10. Network Mode Data Pump Export and Import both support a network mode in which the jobs source is a remote Oracle instance. This is an overlap of unloading the data, using Export, and loading the data, using Import, so those processes dont have to be serialized. A database link is used for the network. We dont have to worry about allocating file space because there are no intermediate dump files. Network Export Unload a remote database to a local dump file set Allows export of read-only databases for archiving Network Import Overlap execution of extract and load No intermediate dump files Because Data Pump maintains a Master Control Table and must perform database writes, Data Pump cant directly Export a Read-only database. Network mode allows the user to export Read-Only databases: The Data Pump Export job runs locally on a read/write instance and extracts the data and metadata from the remote read-only instance. Both networkexport and import use database links to communicate with the remote source. First level parallelism is supported for both network export and import. I/O servers do not operate remotely, so second level, intra-partition parallelism is not supported in network operations. 11.Fine-Grained Object Selection All object types are supported - With the new EXCLUDE and INCLUDE parameters, a Data Pump job can include or exclude any type of object and any subset of objects within a type. Exclude parameter: specified object types are excluded from the operation Include parameter: only the specified object types are included

Both take an optional name filter for even finer granularity: INCLUDE/ PACKAGE: LIKE PAYROLL% EXCLUDE TABLE: in (FOO,BAR,) e.g.:

EXCLUDE=function EXCLUDE=procedure EXCLUDE=package:like PAYROLL% Would exclude all functions, procedures, and packages with names starting with PAYROLL from the job. Using INCLUDE instead of EXCLUDE above, would include the functions, procedures, and packages with names startingwith PAYROLL. 12.DDL Transformations Easy with XML, because object metadata is stored as XML in the dump file set,it is easy to apply transformations when DDL is being formed (via XSL-T) during import. REMAP_SCHEMA -> REMAP_SCHEMA provides the old FROMUSER / TOUSER capability to change object ownership. REMAP_TABLESPACE -> REMAP_TABLESPACE allows objects to be moved from one tablespace to another. This changes the tablespace definition as well REMAP_DATAFILE -> REMAP_DATAFILE is useful when moving databases across platforms that have different file system semantics. Segment and storage attributes can be suppressed -> The TRANSFORM parameter can also be used so that storage clauses are not generated in the DDL. This is useful if the storage characteristics of the target instance are very different from those of the source.

Data Pump Benefits (advantages over normal export & import)

o o o o o o o o o o o o o o o o Restartable Improved control Files will created on server, not on client side Parallel execution Online writing course A Jobs Web Designers Locations Intermediate Packages Automated performance tuning Simplified monitoring Improved object filtering Dump will be compressed Data can be encrypted (in Oracle 11g or later) Remap of data during export or import (in 11g or later)

o We can export one or more partitions of a table without having to move the entire table (in 11g or later) o XML schemas and XMLType columns are supported for both export and import (in 11g or later) o Using the Direct Path method of unloading or loading data, a single stream of Data Pump export (unload) is approximately 2 times faster than original Export , because the Direct Path API has been modified to be even more efficient. Depending on the level of parallelism, the level of improvement can be much more.

o Original Import uses only conventional mode inserts, so a single stream of Data Pump Import is 10-45 times faster than normal Import. As with Export, the jobs single stream can be changed to parallel streams for even more improvement. o With Data Pump, it is much easier for the DBA to manage and monitor jobs. During a long-running job, the DBA can monitor a job from multiple locations and know how far along it is, how much there is left to go, what objects are being worked on, etc. The DBA can also affect the jobs operation, i.e. abort it, adjust its resource consumption, and stop it for later restart. o Since the jobs are completed much more quickly than before, production systems have less downtime. o Data Pump is publicly available as a PL/SQL package (DBMS_DATAPUMP), so customers can write their own data movement utilities if so desired. The metadata capabilities of the Data Pump are also available as a separate PL/SQL package, DBMS_METADATA. o While importing, if destination schema is not existed, Data Pump will create the user and import the objects. Data Pump requires no special tuning. Data Pump runs optimally out of the box. Original Export and (especially) Import require careful tuning to achieve optimum results. There are no Data Pump performance tuning parameters other than the ability to dynamically adjust the degree of parallelism. We can dynamically throttle the number of threads of execution throughout the lifetime of the job. There is an interactive command mode where we can adjust the level of parallelism. For example, we can start up a job during the day with a PARALLEL=2, and then increase it at night to a higher level. All the Oracle database data types are supported via Data Pumps two data movement mechanisms, Direct Path and External Tables. With Data Pump, there is much more flexibility in selecting objects for unload and load operations. We can now unload any subset of database objects (such as functions, packages, and procedures) and reload them on the target platform. Almost all database object types can be excluded or included in an operation using the new Exclude and Include parameters. We can either use the Command line interface or the Oracle Enterprise Manager web-based GUI interface. Data Pump handles all the necessary compatibility issues between hardware platforms and operating systems. Oracle Data Pump supports Oracle Apps 11i. We can use the ESTIMATE ONLY command to see how much disk space is required for the jobs dump file set before we start the operation. Jobs can be monitored from any location is going on. Clients may also detach from an executing job without affecting it. Every Data Pump job creates a Master Table in which the entire record of the job is maintained. The Master Table is the directory to the job, so if a job is stopped for any reason, it can be restarted at a later point in time, without losing any data. Whenever Data Pump export or import is running, Oracle will create a table with the JOB_NAME and will be deleted once the job is done. From this table, Oracle will find out how much job has been completed and from where to continue etc. With Data Pump, it is now possible to change the definition of some objects as they are created at import time. For example, we can remap the source datafile name to the target datafile name in all DDL statements where the source datafile is referenced. This is really useful if we are moving across platforms

with different file system syntax. Data Pump supports the Flashback infrastructure, so we can perform an export and get a dumpfile set that is consistent with a specified point in time or SCN.

Data Pump Vs SQL*Loader

We can use SQL*Loader to load data from external files into tables of an Oracle database. Many customers use SQL*Loader on a daily basis to load files (e.g. financial feeds) into their databases. Data Pump Export and Import may be used less frequently, but for very important tasks, such as migrating between platforms, moving data between development, test, and production databases, logical database backup, and for application deployment throughout a corporation.

Data Pump Vs Transportable Tablespaces

We can use Transportable Tablespaces when we want to move an entire tablespace of data from one Oracle database to another. Transportable Tablespaces allows Oracle data files to be unplugged from a database, moved or copied to another location, and then plugged into another database. Moving data using Transportable Tablespaces can be much faster than performing either an export or import of the same data, because transporting a tablespace only requires the copying of datafiles and integrating the tablespace dictionary information. Even when transporting a tablespace, Data Pump Export and Import are still used to handle the extraction and recreation of the metadata for that tablespace.

The various background processes related to Data Pump are: Data Pump Master Process (DMnn) : Creates and deletes the master table at the time of export and import. Master table contains the job state and object information. Coordinates the Data Pump job tasks performed by Data Pump worker processes and handles client interactions. The Data Pump master (control) process is started during job creation and coordinates all tasks performed by the Data Pump job. It handles all client interactions and communication, establishes all job contexts, and coordinates all worker process activities on behalf of the job. Creates the Worker Process. Data Pump Worker Process (DWnn) : It performs the actual heavy duty work of loading and unloading of data. It maintains the information in master table. The Data Pump worker process is responsible for performing tasks that are assigned by the Data Pump master process, such as the loading and unloading of metadata and data. Shadow Process : When client logs in to an Oracle Server the database creates and Oracle process to service Data Pump API. Client Process : The client process calls the Data pump API.

Data Pump Disadvantages

Cant use UNIX pipes Can't run as SYS (/ as sysdba)

How Data Pump accesses loading and unloading of Data:

Oracle has provided direct path to unload or export operations since Oracle 7.3. This method has been very useful for DBAs that want a quick export of the database and this process has been further enhanced in the Data Pump technology. Oracle uses the direct path method for loading (impdp) and unloading (expdp) when the structure of the tables allows it. If the table is part of a cluster, or it has a global index on a partitioned table, then Data Pump accesses the data in a different method called External Table. Both the direct path load and external table method support the same external data representation, so we can load the data that was unloaded with External Table method and vice versa.

Getting Started...
As stated earlier, Data Pump is a server-based utility, rather than client-based; dump files, log files, and SQL files are accessed relative to server-based directory paths. Data Pump requires you to specify directory paths as directory objects. A directory object maps a name to a directory path on the file system. 1. The following SQL statements creates a user, a directory object named dpump_dir1 and grants the permissions to the user. SQLPLUS system/manager@TDB10G as sysdba SQL> create user dpuser identified by dpuser; SQL> grant connect, resource to dpuser; SQL> CREATE DIRECTORY dpump_dir1 AS '/opt/app/oracle'; SQL> grant read, write on directory dpump_dir1 to dpuser 2. Let us see how the INCLUDE and EXCLUDE parameters can be used to limit the load and unload of particular objects. When the INCLUDE parameter is used, only the objects specified by it will be included in the export. When the EXCLUDE parameter is used, all objects except those specified by it will be included in the export: Assume we have EMP,EMP_DETAILS and DEPT tables owned by dpuser. $ expdp dpuser/dpuser@TDB10G schemas=dpuser include= TABLE:\"IN (\'EMP\', \'DEPT\')\" directory=dpump_dir1 dumpfile=dpuser.dmp log=dpuser.log $expdp dpuser/dpuser@TDB10G schemas=dpuser exclude=TABLE:\"= \'EMP_DETAILS\'\" directory=dpump_dir1 dumpfile=dpuser2.dmp logfile=dpuser.log As stated earlier, Data pump performance can be significantly improved by using thePARALLEL parameter. This should be used in conjunction with the "%U" wildcard in theDUMPFILE parameter to allow multiple dumpfiles to be created or read: $expdp dpuser/dpuser@TDB10G schemas=dpuser directory=dpump_dir1 parallel=4 dumpfile=dpuser_%U.dmp logfile=dpuser.log

Data Pump API:

The Data Pump API, DBMS_DATAPUMP, provides a high-speed mechanism to move the data from one database to another. Infact, the Data Pump Export and Data Pump Import utilities are based on the Data Pump API. The structure used in the client interface of this API is a job handle. Job handle can be created using the OPEN or ATTACH function of the DBMS_DATAPUMP package. Other DBA sessions can attach to a job to monitor and control its progress so that remote DBA can monitor the job that was scheduled by an on-site DBA. The following steps list the basic activities involved in using Data Pump API.

1. 2. 3.

Execute DBMS_DATAPUMP.OPEN procedure to create job. Define parameters for the job like adding file and filters etc. Start the job.

4. 5. 6. 7.

Optionally monitor the job until it completes. Optionally detach from job and attach at later time. Optionally, stop the job Restart the job that was stopped.

Example of the above steps: Declare P_handle number; --- -- Data Pump job handle P_last_job_state varchar2(45); ---- -- To keep track of job state P_job_state varchar2(45); P_status ku$_Status ----- -- The status object returned by get_status BEGIN P_handle:=DBMS_DATAPUMP.OPEN ('EXPORT','SCHEMA', NULL,'EXAMPLE','LATEST'); -- Specify a single dump file for the job (using the handle just returned) -- and a directory object, which must already be defined and accessible -- to the user running this procedure DBMS_DATAPUMP.ADD_FILE (p_handle,'example.dmp','DMPDIR'); -- A metadata filter is used to specify the schema that will be exported. DBMS_DATAPUMP.METADATA_FILTER (p_handle,'SCHEMA_EXPR','IN (''dpuser'')'); -- Start the job. An exception will be generated if something is not set up -- Properly. DBMS_DATAPUMP.start_job (p_handle); ----The export job should now be running. The status of the job can be checked by writing a separate procedure and capturing the errors and status until it is completed. Overall job status can also be obtained by querying SELECT * from dba_datapump_jobs.

Oracle Data Pump is a great tool for the fast movement of data between the databases and much of this performance improvement is derived from the use of parameter parallelism. Even when the Transportable Tablespace feature of Oracle is used to move self-contained data between the databases, Data Pump is still required for handling the extraction and recreation of the metadata for that tablespace. Whenever possible, Data Pump performance is further maximized by using Direct-Path driver. Otherwise, Data Pump accesses the data using an External Table access driver.Data Pump provides flexibility, with the implementation of parameters such as INCLUDE, EXCLUDE, QUERY, and TRANSFORM that gives the DBA more control of data and objects being loaded and unloaded. With all of these features, Data Pump is a welcome addition to DBA tools in a world that constantly redefines the size of the large database.

How Does Data Pump Access Data?

Data Pump supports two access methods to load and unload table row data: direct path and external tables. Because both methods support the same external data representation, data that is unloaded with one method can be loaded using the other method. Data Pump automatically chooses the fastest method appropriate for each table. Data Pump also uses functionality provided in the DBMS_METADATA PL/SQL package to handle all operations involving metadata, including complete extraction, transformation, and re-creation of all database object definitions.
Direct Path Loads and Unloads

The Oracle database has provided direct path unload capability for export operations since Oracle release 7.3 and a direct path API for OCI since Oracle8i. Data Pump technology enhances direct path technology in the following ways:

Support of a direct path, proprietary format unload. Improved performance through elimination of unnecessary conversions. This is possible because the direct path internal stream format is used as the format stored in the Data Pump dump files. Support of additional datatypes and transformations.

Data Pump uses direct path load and unload when the structure of a table allows it. In the following circumstances, Data Pump cannot use direct path loading:

A global index on multipartition tables exists during a single-partition load. This includes object tables that are partitioned. A domain index exists for a LOB column. A table is in a cluster. A table has an active trigger. A table has fine-grained access control enabled in insert mode. A table contains BFILE columns or columns of opaque types. A referential integrity constraint is present. A table contains VARRAY columns with an embedded opaque type.

If any of these conditions exist for a table, Data Pump uses external tables rather than direct path to move the data for that table.
External Tables

The Oracle database has provided an external tables capability since Oracle9 i that allows reading of data sources external to the database. As of Oracle Database 10g, the external tables feature also supports writing database data to destinations external to the database. Data Pump provides an external tables access driver(ORACLE_DATAPUMP) that reads and writes files. The format of the files is the same format used with the direct path method. This allows for high-speed loading and

unloading of database tables. Data Pump uses external tables as the data access mechanism in the following situations:

Loading and unloading very large tables and partitions in situations where parallel SQL can be used to advantage Loading tables with global or domain indexes defined on them, including partitioned object tables Loading tables with active triggers or clustered tables Loading and unloading tables with encrypted columns Loading tables with fine-grained access control enabled for inserts Loading tables that are partitioned differently at load time and unload time

Query Parameter in Datapump (EXPDP/IMPDP) utility Sometimes we need to export or imprt only some data from a huge table. This can be done by using query parameter in Datapump utility. Query parameter can be used in both EXPDP as well as IMPDP utility. 1. EXPDP :Query parameter can be used in two ways :a. Using parameter file :bash-3.2$ bash-3.2$ cat test.par TABLES=AMIT directory=datapump DUMPFILE=expdp_test.dmp logfile=expdp_test.log query=amit:"where DATE > '01-DEC-2013'" bash-3.2$ bash-3.2$ expdp user/user parfile=test.par Export: Release - Production on Fri Dec 13 12:20:47 2013 Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options Starting "USER"."SYS_EXPORT_TABLE_01": user/****** parfile=test.par Estimate in progress using BLOCKS method... Processing object type TABLE_EXPORT/TABLE/TABLE_DATA Total estimation using BLOCKS method: 384 KB Processing object type TABLE_EXPORT/TABLE/TABLE Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS . . exported "USER"."AMIT" 30.67 KB 75 rows Master table "USER"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded ****************************************************************************** Dump file set for USER.SYS_EXPORT_TABLE_01 is: /disk1/datapump/expdp_etst.dmp Job "USER"."SYS_EXPORT_TABLE_01" successfully completed at 12:20:53 b. In Command Line :expdp user/user directory=datapump dumpfile=expdp_taa_01.dmp logfile=expdp_taa_01.log query=amit:\"where DATE \> \'01-DEC-2013\'\" tables=AMIT Export: Release - Production on Fri Dec 13 12:24:50 2013

Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options Starting "USER"."SYS_EXPORT_TABLE_01": user/******** directory=datapump dumpfile=expdp_taa_01.dmp logfile=expdp_taa_01.log query=amit:"where DATE > '01-DEC-2013'" tables=AMIT Estimate in progress using BLOCKS method... Processing object type TABLE_EXPORT/TABLE/TABLE_DATA Total estimation using BLOCKS method: 384 KB Processing object type TABLE_EXPORT/TABLE/TABLE Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS . . exported "USER"."AMIT" 30.67 KB 75 rows Master table "USER"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded ****************************************************************************** Dump file set for USER.SYS_EXPORT_TABLE_01 is: /disk1/datapump/expdp_taa_01.dmp Job "USER"."SYS_EXPORT_TABLE_01" successfully completed at 12:24:56 2. IMPDP :a. Using par file :bash-3.2$ cat test.par schemas=DATA include=TABLE:"IN('TEST')" directory=datapump DUMPFILE=expdp_fullDB_12_Dec_13.dmp logfile=expdp_etst.log query=test:"where DATE > '01-DEC-2013'" remap_schema=DATA:AMIT bash-3.2$ impdp amit/amit parfile=test.par Import: Release - Production on Fri Dec 13 12:44:22 2013 Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options Master table "AMIT"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded Starting "AMIT"."SYS_IMPORT_SCHEMA_01": amit/******** parfile=test.par Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA . . imported "AMIT"."TEST" 51.17 KB 75 out of 1614 rows Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/INDEX


bash-3.2$ impdp amit/amit directory=datapump dumpfile=expdp_fullDB_12_Dec_13.dmp logfile=test.log query=TEST:\"where DATE \> \'01-DEC-2013\'\" schemas= DATA include=TABLE:\"IN\(\'TEST\'\)\" remap_schema=DATA:AMIT

Import: Release - Production on Fri Dec 13 12:38:29 2013 Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options Master table "AMIT"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded Starting "AMIT"."SYS_IMPORT_SCHEMA_01": amit/******** directory=datapump dumpfile=expdp_fullDB_12_Dec_13.dmp logfile=test.log query=test:"where DATE > '01-DEC-2013'" schemas=DATA include=TABLE:"IN('TEST')" remap_schema=DATA:AMIT Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA . . imported "AMIT"."TEST" 51.17 KB 75 out of 1614 rows Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/INDEX Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/STATISTICS/INDEX_STATISTICS Processing object type DATABASE_EXPORT/SCHEMA/TABLE/STATISTICS/TABLE_STATISTICS Job "AMIT"."SYS_IMPORT_SCHEMA_01" successfully completed at 12:38:55

The Bad File The bad file contains records that were rejected, either by SQL*Loader or by the Oracle database. If you do not specify a bad file and there are rejected records, then SQL*Loader automatically creates one. It will have the same name as the data file, with a.bad extension. Some of the possible reasons for rejection are discussed in the next sections. The Discard File As SQL*Loader executes, it may create a file called the discard file. This file is created only when it is needed, and only if you have specified that a discard file should be enabled. The discard file contains records that were filtered out of the load because they did not match any record-selection criteria specified in the control file. The discard file therefore contains records that were not inserted into any table in the database. You can specify the maximum number of such records that the discard file can accept. Data written to any database table is not written to the discard file.

What Is a Control File?

Every Oracle Database has a control file, which is a small binary file that records the physical structure of the database. The control file includes:

The database name Names and locations of associated datafiles and redo log files The timestamp of the database creation The current log sequence number Checkpoint information

The control file must be available for writing by the Oracle Database server whenever the database is open. Without the control file, the database cannot be mounted and recovery is difficult. The control file of an Oracle Database is created at the same time as the database. By default, at least one copy of the control file is created during database creation. On some operating systems the default is to create multiple copies. You should create two or more copies of the control file during database creation. You can also create control files later, if you lose control files or want to change particular settings in the control files.

SQL*Loader Maximazing SQL*Loader Performance Use Direct Path Loads - The conventional path loader essentially loads the data by using standard insert statements. The direct path loader (direct=true) loads directly into the Oracle data files and creates blocks in Oracle database block format. There are certain cases, however, in which direct path loads cannot be used (clustered tables). To prepare the database for direct path loads, the script $ORACLE_HOME/rdbms/admin/catldr.sql.sql must be executed. Disable Indexes and Constraints. For conventional data loads only, the disabling of indexes and constraints can greatly enhance the performance. Use a Larger Bind Array. For conventional data loads only, larger bind arrays limit the number of calls to the database and increase performance. The size of the bind array is specified using the bindsize parameter. The bind array's size is equivalent to the number of rows it contains (rows=) times the maximum length of each row. Use ROWS=n to Commit Less Frequently. For conventional data loads only, the rows parameter specifies the number of rows per commit. Issuing fewer commits will enhance performance. Use Parallel Loads. Available with direct path data loads only, this option allows multiple SQL*Loader jobs to execute concurrently. $ sqlldr control=first.ctl parallel=true direct=true $ sqlldr control=second.ctl parallel=true direct=true

Use Fixed Width Data. Fixed width data format saves Oracle some processing when parsing the data. The savings can be tremendous. Disable Archiving During Load. While this may not be feasible in certain environments, disabling database archiving can increase performance considerably. Use unrecoverable. The unrecoverable option (unrecoverable load data) disables the writing of the data to the redo logs. This option is available for direct path loads only. Using the table table_with_one_million_rows, the following benchmark tests were performed with the various SQL*Loader options. The table was truncated after each test. SQL*Loader Option Elapsed Time (Seconds) Time Reduction direct=false 135 rows=64 direct=false 92 32% bindsize=512000 rows=10000 direct=false 85 37% bindsize=512000 rows=10000 database in noarchivelog direct=true 47 65%

SQL*Loader Option direct=true unrecoverable direct=true unrecoverable fixed width data

Elapsed Time (Seconds) Time Reduction 41 70% 41 70%

The results above indicate that conventional path loads take the longest. However, the bindsize and rows parameters can aid the performance under these loads. The test involving the conventional load didnt come close to the performance of the direct path load with the unrecoverable option specified. It is also worth noting that the fastest import time achieved for this table (earlier) was 67 seconds, compared to 41 for SQL*Loader direct path a 39% reduction in execution time. This proves that SQL*Loader can load the same data faster than import. These tests did not compensate for indexes. All database load operations will execute faster when indexes are disabled.

SQL*Loader Control File The control file is a text file written in a language that SQL*Loader understands. The control file describes the task that the SQL*Loader is to carry out. The control file tells SQL*Loader where to find the data, how to parse and interpret the data, where to insert the data, and more. See Chapter 4, "SQL*Loader Case Studies" for example control files. Although not precisely defined, a control file can be said to have three sections: 1. The first section contains session-wide information, for example: o global options such as bindsize, rows, records to skip, etc. o INFILE clauses to specify where the input data is located o data character set specification 2. The second section consists of one or more "INTO TABLE" blocks. Each of these blocks contains information about the table into which the data is to be loadedsuch as the table name and the columns of the table. 3. The third section is optional and, if present, contains input data.