Wednesday, June 20, 2018

How to redirect the oracle golden gate output to another files

Here is the simple code to redirect the  OGG output to new text file
you need to add the command on the text file and need to pass on this script.

cat obey_cmds.txt

info all



echo obey /appods/ggshp/v11110078/obey_cmds.txt | $GGS_HOME/ggsci > Replication_info_all.log

Wednesday, June 13, 2018

Nz_migrate throws the error


Issue:
when I tried to migrate the date from source to target ,got the below error and looks like all syntax and everything correct,even though got the below error.

nz@<netezza_server_name>:nz@NPS_HOST:/export/home/nz> nz_migrate -shost NPS_HOST -thost NPS_HOST -sdb DBA_DB -tdb NPS_DB_OLD  -tableFile  /export/home/nz/NPS_DB_OLD_tbl.log  -suser admin -spassword  ****** -tuser admin -tpassword ****** -CreateTargetTable YES -TruncateTargetTable NO -format binary  -threads 4

-bash: !0: event not found

Solution:
just use single quote on the password character,it will  run without any issues.double  quote wont work.

nz_migrate -shost NPS_HOST -thost NPS_HOST -sdb DBA_DB -tdb NPS_DB_OLD  -tableFile  /export/home/nz/NPS_DB_OLD_tbl.log  -suser admin -spassword  '****** '-tuser admin -tpassword '******' -CreateTargetTable YES -TruncateTargetTable NO -format binary  -threads 4

Monday, June 11, 2018

Step by step troubleshooting for Netezza database down

Here is the steps  troubleshoot for Netezza database down issue

1.look for the  contents of sysmgr.log under/nz/kit/log/sysmgr
2.look for the content of the pg.log under /nz/ki/log/postgres/
3.grep the timeline of the DB down from the pg.log


nz@<server_name>:/export/home/nz> grep -i "shut down" /nz/kit/log/postgres/pg.log.1
2018-06-05 05:07:16.858990 EDT [30134]  NOTICE:  database system was shut down at 2018-06-05 05:07:16 EDT
2018-06-05 14:00:39.323065 EDT [1983]   NOTICE:  database system was shut down at 2018-06-05 14:00:39 EDT
2018-06-05 16:29:28.631563 EDT [17920]  NOTICE:  database system was shut down at 2018-06-05 16:29:28 EDT
2018-06-05 17:21:01.303872 EDT [27842]  NOTICE:  database system was shut down at 2018-06-05 17:21:01 EDT

2018-06-05 18:35:51.314193 EDT [2921]   NOTICE:  database system was shut down at 2018-06-05 18:35:51 EDT

4.look for the core files and convert the core files to readable format.


  456  2018-06-05 23:56:17 nzgdb -core /nz/data/base/1/core.1528233612.21424 -bt > 21424.bt
  457  2018-06-05 23:56:51 nzgdb -core /nz/data/base/1/core.1528230523.11816 -bt > 11816.bt
  458  2018-06-05 23:57:29 nzgdb -core /nz/data/base/1/core.1528221590.28144 -bt > 28144.bt
  459  2018-06-05 23:58:03 nzgdb -core /nz/data/base/1/core.1528189585.23904 -bt > 23904.bt

  460  2018-06-05 23:58:46 nzgdb -core /nz/data/base/1/core.1528185575.5528 -bt > 5528.bt

5.grep the PID with pg.log find out what went wrong on the system during the core dump generated time.


  466  2018-06-06 00:05:22 grep '\[27208\]' /nz/kit/log/postgres/pg.log

  467  2018-06-06 00:09:44 2018-06-05 18:34:36.874070 EDT [27208]  DEBUG:  connection: host=10.128.72.44 user=NZKOMUSER database=CVKOMNZP remotepid=5177762 fetype=1

  468  2018-06-06 00:09:44 2018-06-05 18:34:36.874088 EDT [27208]  DEBUG:  Session id is 16279

  469  2018-06-06 00:09:44 2018-06-05 18:34:36.877957 EDT [27208]  DEBUG:  QUERY: select current_catalog, current_user

  470  2018-06-06 00:09:44 2018-06-05 18:34:36.879496 EDT [27208]  DEBUG:  QUERY: exec dta_box_inv_hist_incr_load(' TARGET_TABLE==DTA_STG_nz_tbl_HISTORY><TARGET_FINAL_TABLE==nz_tbl_HISTORY><SRC_LAST_MODIFIED_DATE_COL==dtm_last_updated><SRC_PULL_DATE_COL==BOXINVTRY_ID><SRC_PARALLEL_TASK_COL==><SOURCE_TABLE==nz_tbl_HISTORY><SOURCE_SCHEMA==KOMUSER><SOURCE_FILTER==><SRC_PK_COL==BOXINVTRY_ID><DTA_JOB_ID==503><TPPIL_KLMDATE==04-jun-2018 06:49:29 AM ')

  471  2018-06-06 00:09:44 2018-06-05 18:34:36.897353 EDT [27208]  DEBUG:  UpdateStatsInsertDummy called on table 'TEMPFUNC13279476'

  472  2018-06-06 00:09:44 2018-06-05 18:34:36.906701 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: SELECT  dta_nz_parse_parameter_str(ltrim( $1 )) - Variables ( TARGET_TABLE==DTA_STG_nz_tbl_HISTORY><TARGET_FINAL_TABLE==nz_tbl_HISTORY><SRC_LAST_MODIFIED_DATE_COL==dtm_last_updated><SRC_PULL_DATE_COL==BOXINVTRY_ID><SRC_PARALLEL_TASK_COL==><SOURCE_TABLE==nz_tbl_HISTORY><SOURCE_SCHEMA==KOMUSER><SOURCE_FILTER==><SRC_PK_COL==BOXINVTRY_ID><DTA_JOB_ID==503><TPPIL_KLMDATE==04-jun-2018 06:49:29 AM )

  473  2018-06-06 00:09:44 2018-06-05 18:34:36.914496 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: SELECT  * from table(REGEXP_SPLIT_ROWS( $1 ,'><')) - Variables (TARGET_TABLE==DTA_STG_nz_tbl_HISTORY><TARGET_FINAL_TABLE==nz_tbl_HISTORY><SRC_LAST_MODIFIED_DATE_COL==dtm_last_updated><SRC_PULL_DATE_COL==BOXINVTRY_ID><SRC_PARALLEL_TASK_COL==><SOURCE_TABLE==nz_tbl_HISTORY><SOURCE_SCHEMA==KOMUSER><SOURCE_FILTER==><SRC_PK_COL==BOXINVTRY_ID><DTA_JOB_ID==503><TPPIL_KLMDATE==04-jun-2018 06:49:29 AM )

  474  2018-06-06 00:09:44 2018-06-05 18:34:36.927852 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'TARGET_TABLE','DTA_STG_nz_tbl_HISTORY')

  475  2018-06-06 00:09:44 2018-06-05 18:34:41.076371 EDT [27208]  DEBUG:  1 rows inserted

  476  2018-06-06 00:09:44 2018-06-05 18:34:41.084061 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'TARGET_FINAL_TABLE','nz_tbl_HISTORY')

  477  2018-06-06 00:09:44 2018-06-05 18:34:44.982932 EDT [27208]  DEBUG:  1 rows inserted

  478  2018-06-06 00:09:44 2018-06-05 18:34:44.990653 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'SRC_LAST_MODIFIED_DATE_COL','dtm_last_updated')

  479  2018-06-06 00:09:44 2018-06-05 18:34:47.670894 EDT [27208]  DEBUG:  1 rows inserted

  480  2018-06-06 00:09:44 2018-06-05 18:34:47.678735 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'SRC_PULL_DATE_COL','BOXINVTRY_ID')

  481  2018-06-06 00:09:44 2018-06-05 18:34:52.229912 EDT [27208]  DEBUG:  1 rows inserted

  482  2018-06-06 00:09:44 2018-06-05 18:34:52.237441 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'SRC_PARALLEL_TASK_COL','')

  483  2018-06-06 00:09:44 2018-06-05 18:34:53.913637 EDT [27208]  DEBUG:  1 rows inserted

  484  2018-06-06 00:09:44 2018-06-05 18:34:53.926314 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'SOURCE_TABLE','nz_tbl_HISTORY')

  485  2018-06-06 00:09:44 2018-06-05 18:34:55.508722 EDT [27208]  DEBUG:  1 rows inserted

  486  2018-06-06 00:09:44 2018-06-05 18:34:55.518331 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'SOURCE_SCHEMA','KOMUSER')

  487  2018-06-06 00:09:44 2018-06-05 18:34:56.975557 EDT [27208]  DEBUG:  1 rows inserted

  488  2018-06-06 00:09:44 2018-06-05 18:34:56.983190 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'SOURCE_FILTER','')

  489  2018-06-06 00:09:44 2018-06-05 18:34:59.367628 EDT [27208]  DEBUG:  1 rows inserted

  490  2018-06-06 00:09:44 2018-06-05 18:34:59.375228 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'SRC_PK_COL','BOXINVTRY_ID')

  491  2018-06-06 00:09:44 2018-06-05 18:35:00.948244 EDT [27208]  DEBUG:  1 rows inserted

  492  2018-06-06 00:09:44 2018-06-05 18:35:00.955864 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'DTA_JOB_ID','503')

  493  2018-06-06 00:09:44 2018-06-05 18:35:04.072782 EDT [27208]  DEBUG:  1 rows inserted

  494  2018-06-06 00:09:44 2018-06-05 18:35:04.080479 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: insert into TEMPFUNC13279476 values( 'TPPIL_KLMDATE','04-jun-2018 06:49:29 AM ')

  495  2018-06-06 00:09:44 2018-06-05 18:35:06.168680 EDT [27208]  DEBUG:  1 rows inserted

  496  2018-06-06 00:09:44 2018-06-05 18:35:06.401591 EDT [27208]  DEBUG:  STORED PROCEDURE EXEC: delete from nz_tbl_

we had checked all the coredump generated during the time all the time NPS executing the same query and crashing the NPS and finally we asked user to stop executing the query and later IBM found that it was bug on the 7.0 version system

work around for this bug:
1)upgrade the NPS to recent version
2)clean the versioning of the table

we cleared the version of the table and NPs stable after that.



unable to start the Netezza database

issue:
we have done some  maintanance on Netezza 7.0 and after maintenance we are unable to start the Netezza database,it throws the below error,while starting the NPS.

nz@<server_name>:/export/home/nz> nzstart
ERROR:  System limit for 'nofile' is too low (1024); must be 9600
ERROR:  System limit for 'nice' is too low (0); must be 20
nz@<server_name>:/export/home/nz> nzstate
System state is 'Stopped'.

Solution:
when we check the log,we dont have anything on sysmgr.log,because ,it not even started HW components like SPU & DISK,Then we relaised the ulimit  value would be small to startup the system and increased the ulimit and issue resolved.

However to increase the ulimit on the Netezza server you may need root privilege to update the new value on the server.

This is happened because our NPS got failover to HA2 and we never  had HA2 as active server and this server does not have sufficient ulimit value and fixed later.

ulimit -n 9600

Friday, June 8, 2018

How to resolve version tables in Netezza database?

whenever an alter happened on the Netezza table and NPS will make that table as version and collect the data from both version which is considered as costlier operation.
So we have to clear the  table versions periodically.

Versioned tables come about as a result of doing an
               ALTER TABLE <tablename> [ADD|DROP] COLUMN ...

          This results in multiple data stores for the table.  When you go to query
          the table, NPS must recombine the separate data stores back into a single
          entity.  This action will be performed automatically and on-the-fly.  But
          it does result in additional query overhead.  Therefore, it is a best
          practice to reconstitute the table as soon as practical by doing a

Here  is the script to find out the table versions in Netezza

/nz> nz_altered_tables 


# Of Versioned Tables         1                                        
     Total # Of Versions      2                                        
                                                                        
 Database |         Table Name         |       Size (Bytes)       | #  
Of Versions                                                            
----------+----------------------------+--------------------------+-----
----------                                                             
NZ_DB1 | TEST_TABLE_HISTORY |          197,591,957,504 |    



once you find the table ,then you need to clear the version using the below command.

 GROOM TABLE <tablename> VERSIONS;

Friday, June 1, 2018

what will happen when you do the full Netezza database restore without specifying the back upset number?

When you try to restore the Netezza database without specifying the back upset ,it will success and it will perform the restore and It will pick the latest backup set from the list and do the restore.

nz@Netezza_Server:/backup1/Netezza/NZ81258-H1/Netezza_DB> ls -ltr
total 24
drwxr-xr-x 3 nz nz 4096 Apr 18 17:53 20180417210222
drwxr-xr-x 3 nz nz 4096 May 14 10:30 20180508000011
drwxr-xr-x 3 nz nz 4096 May 14 10:30 20180501000013
drwxr-xr-x 3 nz nz 4096 May 14 10:30 20180424000009
drwxr-xr-x 3 nz nz 4096 May 14 20:51 20180515000109
drwxr-xr-x 3 nz nz 4096 May 26 23:18 20180522000016


/nz/kit/bin/nzrestore -db  Netezza_DB  -dir /backup1 -lockdb true  -npshost NZ81258-H1

Restore of increment 1 from backupset 20180522000016 to database 'Netezza_DB' committed.

from the above demo,we have 6 backup set with full backup on database,but when we do the Netezza database restore,it picked the last backup and recent backup set and completed the restore.

what will happen when you tries to restore the database and DB name already exist on the netezza server?

I did small test on  restore.
I was trying to do the restore of the database which name already exist on the server and that DB has the same content .

when I did the restore with same name it throws the error and Netezza did not allow the restore.So when you want to restore the  database on Netezza ,just make sure the DB name should not have conflict.



nz@Netezza_Server:/backup1/Netezza/NZ81258-H1/Netezza_DB> /nz/kit/bin/nzrestore -db  Netezza_DB  -dir /backup1 -lockdb true -backupset 20180522000016 -npshost NZ81258-H1
Error: Creation of database failed: Netezza_DB, SQL Error: 'ERROR:  CREATE DATABASE: object Netezza_DB already exists as a DATABASE.'.

before  doing the full restore of exiting database,we need to drop or rename the database and apply the restore,NZrestore cant override the restore of database.


steps to collect plan file for SQL in Netezza database.

Here is the steps to generate the plan file for a SQL in netezza database


nzsql -d databasename -f <create file with SQL statement that has issue> -plndir /nz/data/

you need to pass the database name
full sql statement that has the issue  and location for the plan file

once you execute the above command ,it will generate the below files and under these directory you see the plan files.

nz@servername:/nz/data> ls -ltr
total 1540
nz@servername:/nz/data> ls -ltr
total 1540
-rw-r--r-- 1 nz 500    6727 Jan 15 12:33 full_sql.sql
drwx------ 2 nz 500    4096 Jan 15 12:36 45176
drwx------ 2 nz 500    4096 Jan 15 12:36 45177
drwx------ 2 nz 500    4096 Jan 15 12:36 45178
drwx------ 2 nz 500    4096 Jan 15 12:37 45179