step by step golden gate setup for table level replication

step by step golden gate setup for table level replication

The following procedure describes a minimal Oracle Golden Gate configuration. It is intended for research and/or training purposes and is not intended to be a production configuration. Once the basic configuration has been tested, new features can be added incrementally simplifying any troubleshooting.



In the following configuration I have used the following hosts and databases:

Source Target
Hostname SOURCE TARGET
Database Name SOURCE target
Prepare Databases

Enable Forced Logging

Forced logging should be enabled on the source database to ensure that all changes are written to the redo logs. DBCA does not enable force logging during database creation.

To check if FORCE_LOGGING is enabled in the source database use:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> SELECT force_logging FROM v$database;

FORCE_LOGGING
-------------
NO
If not enabled, then enable FORCE_LOGGING using:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> ALTER DATABASE FORCE LOGGING;

Database altered.
Verify that FORCE_LOGGING has been enabled successfully using:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> SELECT force_logging FROM v$database;

FORCE_LOGGING
-------------
YES
The documentation recommends performing a redo log switch to ensure that the change is applied in all subsequent redo.

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> ALTER SYSTEM SWITCH LOGFILE;

System altered.
Enable Minimal Supplemental Logging

If minimal supplemental logging is not enabled, the extract process fails to start. DBCA does not enable minimal supplemental during database creation.

Minimal supplemental logging only needs to be configured on the source database. It may be prudent to enable it on the target database as well.

Check if minimal supplemental logging is currently enabled:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> SELECT supplemental_log_data_min FROM v$database;

SUPPLEMENTAL_LOG_DATA_MIN
-------------------------
NO
If not enabled then configure minimal supplemental logging:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

Database altered.
Verify that minimal supplemental logging is now enabled:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> SELECT supplemental_log_data_min FROM v$database;

SUPPLEMENTAL_LOG_DATA_MIN
-------------------------
YES
Switch the log file again to ensure that all subsequent redo contains minimal supplemental logging:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> ALTER SYSTEM SWITCH LOGFILE;

System altered.
Prepare Test Environment

Create Test User

In order to test the GoldenGate configuration I created a new schema (sender) containing a new table (T1) as follows. The table is derived from DBA_OBJECTS.

In both databases create a user (schema) called sender. For example:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> CREATE USER sender IDENTIFIED BY sender;

User created.
In both databases grant the following permissions to the new user. For example:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> GRANT CONNECT,RESOURCE,DBA TO sender;

Grant succeeded.
As these are test databases, security is not an issue and therefore DBA privilege has been granted to the new user.

Create Test Table

In the source database (SOURCE) create the T1 table using a subset of rows from DBA_OBJECTS

[oracle@SOURCE]$ sqlplus sender/sender
SQL> CREATE TABLE t1 AS
SELECT object_id,owner,object_name,object_type
FROM dba_objects
WHERE object_id <= 10000;

Table created.
In the source database (SOURCE), add a primary key constraint and index to table T1:

[oracle@SOURCE]$ sqlplus sender/sender
SQL> ALTER TABLE t1 ADD CONSTRAINT t1_i1 PRIMARY KEY (object_id);

Table altered.
Start Listener Processes

Before attempting to configure the network, ensure that the listener processes are running on both servers.

[oracle@SOURCE]$ lsnrctl start

[oracle@TARGET]$ lsnrctl start
Configure Network

For this basic configuration, an entry is required in the TNSNAMES.ORA file on each server describing the database on the other server.

On the source database (SOURCE) I added the following entry for the target database to $ORACLE_HOME/network/admin/tnsnames.ora

target =
  (DESCRIPTION=
  (ADDRESS=(PROTOCOL=TCP)(HOST=TARGET)(PORT=1521))
  (CONNECT_DATA=
    (SERVER=DEDICATED)
    (SERVICE_NAME=target)
  )
)
Verify the connection using SQL*Plus. For example:

[oracle@SOURCE]$ sqlplus sender/sender@target
SQL> SELECT name FROM v$database;

NAME
---------
target
On the target database (TARGET), I added the following entry for the SOURCE database to $ORACLE_HOME/network/admin/tnsnames.ora

SOURCE =
(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=SOURCE)(PORT=1521))
  (CONNECT_DATA=
    (SERVER=DEDICATED)
    (SERVICE_NAME=SOURCE)
  )
)
Verify the connection using SQL*Plus. For example:

[oracle@TARGET]$ sqlplus sender/sender@SOURCE
SQL> SELECT name FROM v$database;

NAME
---------
SOURCE
Create Database Links

On each server create a database link to the other database. This is only required to set up the test data. It is not required for GoldenGate.

On the source server (SOURCE) as the sender user, create the following database link:

[oracle@SOURCE]$ sqlplus sender/sender
SQL> CREATE DATABASE LINK target

CONNECT TO sender IDENTIFIED BY sender USING 'target';
On the target server (TARGET) as the sender user, create the following database link:

[oracle@TARGET]$ sqlplus sender/sender
SQL> CREATE DATABASE LINK SOURCE

CONNECT TO sender IDENTIFIED BY sender USING 'SOURCE';
Copy Test Data to Target Database

Create an initial copy of the test data on the target database

In the target database (target) create a copy of the T1 table using the SQL*Plus command:

[oracle@TARGET]$ sqlplus sender/sender
SQL> CREATE TABLE t1 AS SELECT * FROM t1@SOURCE;
In the target database (target), add a primary key constraint and index to table T1:

[oracle@TARGET]$ sqlplus sender/sender
SQL> ALTER TABLE t1 ADD CONSTRAINT t1_i1 PRIMARY KEY (object_id);
In the source database (SOURCE), verify the number of rows in the original table:

[oracle@SOURCE]$ sqlplus sender/sender
SQL> SELECT COUNT(*) FROM t1;

COUNT(*)
--------
9830
In the target database, verify that there is the same number of rows in the new table:

[oracle@TARGET]$ sqlplus receiver/receiver
SQL> SELECT COUNT(*) FROM t1;

COUNT(*)
--------
9830
Note that the actual number of rows in table T1 should be identical for databases SOURCE and target. However, it may vary in other databases

Configure GoldenGate

Download and Install GoldenGate software

The GoldenGate software can be downloaded from downloads.oracle.com or alternatively from the Oracle E-Delivery website. The software should be installed on both servers.

The 11.2.1.0.1 file for Linux x86-64 is ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip

The zip file is around 90MB in size.

On each server, create a new directory for the GoldenGate software:

[oracle@SOURCE]$ mkdir /home/oracle/goldengate
Copy the download file to the new directory and unzip using:

[oracle@SOURCE]$ unzip ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
Archive: ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
inflating: fbo_ggs_Linux_x64_ora11g_64bit.tar
inflating: OGG_WinUnix_Rel_Notes_11.2.1.0.1.pdf
inflating: Oracle GoldenGate 11.2.1.0.1 README.txt
inflating: Oracle GoldenGate 11.2.1.0.1 README.doc
The tar archive is around 230MB in size.

Extract the files from the tar archive using:

[oracle@SOURCE]$ tar xfv fbo_ggs_Linux_x64_ora11g_64bit.tar
Configure Environment Variables

The /home/oracle/.bash_profile should already contain the following environment variables.

export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
export PATH=$ORACLE_HOME/bin:$PATH
On the source server add the following entries to /home/oracle/.bash_profile

export ORACLE_SID=SOURCE
export PATH=/home/oracle/goldengate:$PATH
export LD_LIBRARY_PATH=/home/oracle/goldengate:$ORACLE_HOME/lib
Ensure the environment variables are set using:

[oracle@SOURCE]$ source /home/oracle/.bash_profile
On the target server add the following entries to /home/oracle/.profile

export ORACLE_SID=target
export PATH=/home/oracle/goldengate:$PATH
export LD_LIBRARY_PATH=/home/oracle/goldengate:$ORACLE_HOME/lib
Ensure the environment variables are set using:

[oracle@TARGET]$ source /home/oracle/.bash_profile
Create GoldenGate Subdirectories

On each server create subdirectories for GoldenGate using the CREATE SUBDIRS command. For example:

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> CREATE SUBDIRS
Creating subdirectories under current directory /home/oracle/goldengate

Parameter files /home/oracle/goldengate/dirprm: already exists
Report files /home/oracle/goldengate/dirrpt: created
Checkpoint files /home/oracle/goldengate/dirchk: created
Process status files /home/oracle/goldengate/dirpcs: created
SQL script files /home/oracle/goldengate/dirsql: created
Database definitions files /home/oracle/goldengate/dirdef: created
Extract data files /media/sf_racattack12c/gg/target/targettrail: created
Temporary files /home/oracle/goldengate/dirtmp: created
Stdout files /home/oracle/goldengate/dirout: created
Create the GoldenGate Schema Owner

A new user should be created to own the GoldenGate database objects.

On each server create the GoldenGate schema owner. For example:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> CREATE USER ggusr IDENTIFIED BY ggusr;

User created.
On each server grant DBA role to the GoldenGate schema owner

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> GRANT CONNECT, RESOURCE, DBA TO ggusr;

Grant succeeded.
On each server set the GGSCHEMA in the global parameter file.

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> EDIT PARAMS ./GLOBALS
In this example the parameters file is /home/oracle/goldengate/GLOBALS

Add the following entry:

GGSCHEMA ggusr
Save and close the file

Create GoldenGate Tablespace

On the source server create a new tablespace for the GoldenGate objects. Ensure that AUTOEXTEND is enabled.

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> CREATE TABLESPACE goldengate
DATAFILE '/media/sf_racattack12c/gg/target/goldengate01.dbf' SIZE 100M AUTOEXTEND ON; target


CREATE TABLESPACE goldengate
DATAFILE '/media/sf_12cR1/gg/source/goldengate01.dbf' SIZE 100M AUTOEXTEND ON;  source
Tablespace created.
Set the new tablespace as the default for the GoldenGate user:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> ALTER USER ggusr DEFAULT TABLESPACE goldengate;

User altered.
On the target server create a new tablespace for the GoldenGate objects. Again ensure that AUTOEXTEND is enabled.

[oracle@TARGET]$ sqlplus / as sysdba
SQL> CREATE TABLESPACE goldengate
DATAFILE '/u01/app/oradata/target/goldengate01.dbf'
SIZE 100M
AUTOEXTEND ON;

Tablespace created.
Set the new tablespace as the default for the GoldenGate user:

[oracle@TARGET]$ sqlplus / as sysdba
SQL> ALTER USER ggusr DEFAULT TABLESPACE goldengate;

User altered.
Run Role Setup script

On the source server run the Role setup script. Specify the GoldenGate schema name when prompted.

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> @role_setup

GGS Role setup script

This script will drop and recreate the role GGS_GGSUSER_ROLE
To use a different role name, quit this script and then edit the params.sql script to change the gg_role parameter to the preferred name. (Do not run the script.)
You will be prompted for the name of a schema for the GoldenGate database objects.
NOTE: The schema must be created prior to running this script.
NOTE: Stop all DDL replication before starting this installation.
Enter GoldenGate schema name:ggusr

Wrote file role_setup_set.txt

PL/SQL procedure successfully completed.

Role setup script complete

Grant this role to each user assigned to the Extract, GGSCI, and Manager processes, by using the following SQL command:
GRANT GGS_GGSUSER_ROLE TO
where is the user assigned to the GoldenGate processes.
On the source server grant GGS_GGSUSER_ROLE to the GoldenGate user:

[oracle@SOURCE]$ sqlplus / as sysdba
SQL> GRANT GGS_GGSUSER_ROLE TO ggusr;

Grant succeeded.
Configure Manager Parameters

On both servers configure the MGR parameters:

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> EDIT PARAMS MGR
The above command created the file /home/oracle/goldengate/dirprm/mgr.prm

Add the following parameters to the parameter file:

PORT 7809
DYNAMICPORTLIST 7810-7820
Save and close the parameter file

Configure Extract Parameters

In this example the extract process will be called "ex1"

On the source server, create the parameter file for Extract ex1:

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> EDIT PARAMS ex1
The above command created the file /home/oracle/goldengate/dirprm/ex1.prm

Add the following parameters to the new file:

EXTRACT ex1
USERID ggusr, PASSWORD ggusr
EXTTRAIL /media/sf_12cR1/gg/source/extrail/ex
TABLE sender.t1;
Configure Data Pump Parameters

In this example the Data Pump process will be called dp1

On the source server create the parameter file for Data Pump process dp1:

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> EDIT PARAMS dp1
The above command created the file /home/oracle/goldengate/dirprm/dp1.prm

Add the following parameters to the new file:

EXTRACT dp1
USERID ggusr, PASSWORD ggusr
RMTHOST 192.168.78.52, MGRPORT 7809
RMTTRAIL /media/sf_racattack12c/gg/target/targettrail/rt
TABLE sender.t1;
Save and close the parameter file

Create Check Point Table

The check point table should be created in the target database.

On the target server login as the ggusr user and add the check point table:

[oracle@TARGET]$ ggsci
GGSCI (TARGET) 1> DBLOGIN USERID ggusr, PASSWORD ggusr

Successfully logged into database.

GGSCI (TARGET) 2> ADD CHECKPOINTTABLE ggusr.checkpointtable

Successfully created checkpoint table ggusr.checkpointtable.
The name of the check point table must be added to the GLOBALS file on the target server.

On the target server edit the GLOBALS file

[oracle@TARGET]$ ggsci
GGSCI (TARGET) 1> EDIT PARAMS ./GLOBALS
Add the CHECKPOINTTABLE parameter to the existing file. For example:

GGSCHEMA ggusr
CHECKPOINTTABLE ggusr.checkpointtable
Save and close the GLOBALS parameter file.

Configure Replication Parameters

On the target server create the parameter file for replication process rep1:

[oracle@TARGET]$ ggsci
GGSCI (TARGET) 1> EDIT PARAMS rep1
The above command created the file /home/oracle/goldengate/dirprm/rep1.prm

Add the following parameters to the new file:

REPLICAT rep1
USERID ggusr, PASSWORD ggusr
ASSUMETARGETDEFS
DISCARDFILE /media/sf_racattack12c/gg/target/discardfile, PURGE
MAP sender.t1, TARGET receiver.t1;
Note that the DISCARDFILE parameter includes the PURGE keyword. If PURGE is not specified them the replication process will fail the second time it is started. Alternatively use the APPEND keyword to append output to the existing file.

Configure Supplemental Logging for Replicated Tables

On the source server configure supplemental logging for all tables that will be replicated. In this example there is only one table (T1)

Supplemental logging can be configured by any user that has privileges to modify the underlying database table.

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> DBLOGIN USERID sender, PASSWORD sender
Successfully logged into database.

GGSCI (SOURCE) 2> ADD TRANDATA t1
Logging of supplemental redo data enabled for table sender.T1.
Add the Extract Process

On the source server add the Extract process (ex1)

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> ADD EXTRACT ex1, TRANLOG, BEGIN NOW

EXTRACT added.
Add the Extract Trail

On the source server add the Extract trail (/media/sf_12cR1/gg/source/extrail)

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> ADD EXTTRAIL /media/sf_12cR1/gg/source/extrail/ex, EXTRACT ex1

EXTTRAIL added.
Add the Data Pump Process

On the source server add the Data Pump process (dp1)

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> ADD EXTRACT dp1 EXTTRAILSOURCE /media/sf_12cR1/gg/source/extrail/ex

EXTRACT added.
Add the Data Pump Trail

On the source server add the Data Pump trail (/home/oracle/gg/dirdat/rt). This trail is created on the target server. However, the name is required in order to set up the Data Pump process on the source server.

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> ADD RMTTRAIL /media/sf_racattack12c/gg/target/targettrail/rt, EXTRACT dp1

RMTTRAIL added.
Add the Replication Process

On the target server add the Replication process (rep1)

[oracle@TARGET]$ ggsci
GGSCI (TARGET) 1> ADD REPLICAT rep1, EXTTRAIL /media/sf_racattack12c/gg/target/targettrail/rt

REPLICAT added.
Start GoldenGate

Start Manager

On the source server, start the GoldenGate manager:

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> START MANAGER

Manager started.
On the target server, start the GoldenGate manager:

[oracle@TARGET]$ ggsci
GGSCI (TARGET) 1> START MANAGER

Manager started.
Start Extract Process

On the source server start the Extract (ex1)

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 1> START EXTRACT ex1

Sending START request to MANAGER ...

EXTRACT EX1 starting

Verify that the Extract has started successfully using INFO EXTRACT:

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE) 10> INFO EXTRACT ex1
EXTRACT EX1 Last Started 2013-02-27 12:57 Status RUNNING
Checkpoint Lag 00:00:24 (updated 00:00:05 ago)
Log Read Checkpoint Oracle Redo Logs
2013-02-27 12:57:01 Seqno 6, RBA 30736
SCN 0.0 (0)
The status should be RUNNING.

Start Data Pump Process

On the source server, start the Data Pump (dp1):

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE.com) 3> START EXTRACT dp1

Sending START request to MANAGER ...

EXTRACT DP1 starting
Verify that the Data Pump has started successfully using INFO EXTRACT:

[oracle@SOURCE]$ ggsci
GGSCI (SOURCE.com) 2> INFO EXTRACT dp1
EXTRACT DP1 Last Started 2013-02-27 11:57 Status RUNNING
Checkpoint Lag 00:00:00 (updated 00:00:00 ago)
Log Read Checkpoint File /home/oracle/gg/dirdat/ex000000
First Record RBA 0
The status should be RUNNING.

Start Replication Process

On the target server, start the Replicat process (rep1):

[oracle@TARGET]$ ggsci
GGSCI (TARGET) 1> START REPLICAT rep1
Sending START request to MANAGER ...
REPLICAT REP1 starting
Verify that the Replicat process has started successfully using INFO EXTRACT:

[oracle@TARGET]$ ggsci
GGSCI (TARGET) 2> INFO REPLICAT rep1
REPLICAT REP1 Last Started 2013-02-27 11:58 Status RUNNING
Checkpoint Lag 00:00:00 (updated 00:00:04 ago)
Log Read Checkpoint File /home/oracle/gg/dirdat/rt000000
First Record RBA 0
The status should be RUNNING.

Test Replication

On the source server, check the number of rows in table T1:

[oracle@SOURCE]$ sqlplus sender/sender
SQL> SELECT COUNT(*) FROM t1;

COUNT(*)
----------
9830
On the target server, check the number of rows in table T1:

[oracle@TARGET]$ sqlplus sender/sender
SQL> SELECT COUNT(*) FROM t1;

COUNT(*)
----------
9830
On the source server, add some rows to table T1. For example:

[oracle@SOURCE]$ sqlplus sender/sender
SQL> INSERT INTO t1 (object_id,owner,object_name,object_type)
SELECT object_id,owner,object_name,object_type
FROM dba_objects
WHERE object_id BETWEEN 10001 AND 11000;

1000 rows created

SQL> COMMIT;

Commit complete
On the source server check the number of rows in table T1:

[oracle@SOURCE]$ sqlplus sender/sender
SQL> SELECT COUNT(*) FROM t1;

COUNT(*)
----------
10830
On the target server check the number of rows in table T1:

[oracle@TARGET]$ sqlplus sender/sender
SQL> SELECT COUNT(*) FROM t1;

COUNT(*)
----------
10830
The number of rows should be identical in both tables

This completes the basic configuration.


------------------------------------------------------------

TESTING
------
alter replicat rep1 begin now

INSERT INTO t1 (object_id,owner,object_name,object_type)
SELECT object_id,owner,object_name,object_type
FROM dba_objects
WHERE object_id <= 10000;

 INSERT INTO t2 (object_id,object_name,object_type)
SELECT object_id,object_name,object_type
FROM dba_objects
WHERE object_id BETWEEN 12000 AND 13000;