Using log dump to find operations in a table between interval time - GoldenGate

1-Open the trace file

Logdump 2 >open eb000025
Current LogTrail is /ggshop/bwdhdbpr025/gghome/dirdat/eb000025

2-Set time interval
Logdump 4 >filter starttime 2013-10-28 11:30
Logdump 5 >filter endtime 2013-10-28 11:59

3-Match all tables and show
Logdump 6 >filter match all
Logdump 7 >filter show
Logdump 8 >count

4-Or check for only one table
Logdump 11 >filter filename TABLE_NAME

####################################################
###### Output example ######
####################################################
SCHEMA.TABLE1                       Partition 4
Total Data Bytes             70524
  Avg Bytes/Record            1410
Insert                          16
LargeObject                     34
After Images                    50
####################################################



Best Regards,
Paulo Portugal

Enable all DDL Commands to be Logged in Alert.log file - Oracle Database 12c

With this simple command below you can enable all DDL commands to be logged in alert.log

ALTER SYSTEM SET enable_ddl_logging = TRUE ;


Best Regards,
Paulo Portugal

Veridata Agent problem. Job Hang in one table.

If your veridata job is running forever , check if you are in 11.2.0.0 version.

I got this error on my veridata agent log:

ERROR:2013-10-24 16:03:50,524 (Session: 1098) - An unexpected exception has occurred: null [RowHashQuery:getCompletion, line 315]
java.lang.ArrayIndexOutOfBoundsException
        at com.goldengate.veridata.db.OracleDatabase$OracleRawTimestampColumnMapper.getBytes(OracleDatabase.java:841)
        at com.goldengate.veridata.db.BaseQuery.packRow(BaseQuery.java:134)
        at com.goldengate.veridata.db.RowHashQuery.call(RowHashQuery.java:112)
        at com.goldengate.veridata.db.RowHashQuery.call(RowHashQuery.java:34)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
        at java.util.concurrent.FutureTask.run(FutureTask.java:149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
        at java.lang.Thread.run(Thread.java:738)


--Solution
Apply patch 11.2.0.1  on all your veridata agents.


Best Regards,
Paulo Portugal

GoldenGate JAgent not starting - FIXED!

Trying to start jagent got this error below:

[hostname]/ggshop/bwdhdbpr052/gghome>  java -jar -Xms64m -Xmx512m dirjar/jagent.jar
0    [main] INFO  com.goldengate.monitor.jagent.JAgentWSMain  - About to call initialize on the WebService
997  [main] INFO  com.goldengate.monitor.jagent.config.impl.AgentInfoImpl  - Using IP address 10.3.0.33 to connect to the Manager Web Service
1002 [main] ERROR com.goldengate.monitor.jagent.JAgentWSMain  - Error Initializing JAgent. JAgent will not be initialized and about to return. Exception: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentBootstrapper': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentInfo': Invocation of init method failed; nested exception is java.lang.NumberFormatException: For input string: "number"
1002 [main] INFO  com.goldengate.monitor.jagent.JAgentWSMain  - JAgentWSMain is about to terminate its operation
1003 [Shutdown Thread.] INFO  com.goldengate.monitor.jagent.JAgentWSMain  - JAgent is about to shutdown.
1004 [Shutdown Thread.] INFO  com.goldengate.monitor.jagent.JAgentWSMain  - JAgent shutdown process is completed successfully.

--Solution
1-Edit mgr.prm file and remove all comments before the line that specify the Manager port:
--Port of Manager -------REMOVE THIS LINE
"port 7809"


Best Regards,
Paulo Portugal

Using kfod to check ASM disks

Checking disks in OS:

[XXXXHOST]/dev> kfod a='/dev/rhdisk*' d=all
--------------------------------------------------------------------------------
 Disk          Size Path
================================================================================
   1:     102750 Mb /dev/rhdisk124
   2:     102750 Mb /dev/rhdisk125
   3:     102750 Mb /dev/rhdisk126
   4:     102750 Mb /dev/rhdisk127
   5:     102750 Mb /dev/rhdisk128
   6:     102750 Mb /dev/rhdisk129
   7:     102750 Mb /dev/rhdisk130
   8:     102750 Mb /dev/rhdisk131
   9:     102750 Mb /dev/rhdisk132
  10:     102750 Mb /dev/rhdisk133
  11:     102750 Mb /dev/rhdisk134
  12:     102750 Mb /dev/rhdisk135
  13:     102750 Mb /dev/rhdisk136
  14:     102750 Mb /dev/rhdisk137
  15:     102750 Mb /dev/rhdisk138
  16:     102750 Mb /dev/rhdisk139
  17:     102750 Mb /dev/rhdisk140
  18:     102750 Mb /dev/rhdisk141
  19:     102750 Mb /dev/rhdisk142
  20:     102750 Mb /dev/rhdisk143
  21:     102750 Mb /dev/rhdisk144
  22:     102750 Mb /dev/rhdisk145
  23:     102750 Mb /dev/rhdisk146
  24:     102750 Mb /dev/rhdisk147
  25:     102750 Mb /dev/rhdisk148
  26:     102750 Mb /dev/rhdisk149
  27:     102750 Mb /dev/rhdisk150
  28:     102750 Mb /dev/rhdisk151
  29:     102750 Mb /dev/rhdisk152
  30:     102750 Mb /dev/rhdisk153
  31:     102750 Mb /dev/rhdisk154
  32:     102750 Mb /dev/rhdisk155
  33:     102750 Mb /dev/rhdisk156
  34:     102750 Mb /dev/rhdisk157
  35:     102750 Mb /dev/rhdisk158
  36:     102750 Mb /dev/rhdisk159
  37:     102750 Mb /dev/rhdisk160
  38:     102750 Mb /dev/rhdisk161
  39:     102750 Mb /dev/rhdisk162
  40:     102750 Mb /dev/rhdisk163
  41:     102750 Mb /dev/rhdisk164
  42:     102750 Mb /dev/rhdisk165
  43:     102750 Mb /dev/rhdisk166
  44:     102750 Mb /dev/rhdisk167
  45:     102750 Mb /dev/rhdisk168
  46:     102750 Mb /dev/rhdisk169
  47:     102750 Mb /dev/rhdisk170
  48:     102750 Mb /dev/rhdisk171
  49:     102750 Mb /dev/rhdisk172
  50:     102750 Mb /dev/rhdisk173
  51:     102750 Mb /dev/rhdisk174
  52:     102750 Mb /dev/rhdisk175
  53:     102750 Mb /dev/rhdisk176
  54:     102750 Mb /dev/rhdisk177
  55:     102750 Mb /dev/rhdisk178
  56:     102750 Mb /dev/rhdisk179
  57:     102750 Mb /dev/rhdisk180
  58:     102750 Mb /dev/rhdisk181
  59:     102750 Mb /dev/rhdisk182
  60:     102750 Mb /dev/rhdisk183
  61:     102750 Mb /dev/rhdisk184
  62:     102750 Mb /dev/rhdisk185
  63:     102750 Mb /dev/rhdisk186
  64:     102750 Mb /dev/rhdisk187
  65:     102750 Mb /dev/rhdisk188
  66:     102750 Mb /dev/rhdisk189
  67:     102750 Mb /dev/rhdisk190
  68:     102750 Mb /dev/rhdisk191
  69:     102750 Mb /dev/rhdisk192
  70:     102750 Mb /dev/rhdisk193
  71:     205500 Mb /dev/rhdisk26
  72:     205500 Mb /dev/rhdisk27
  73:     205500 Mb /dev/rhdisk28
  74:     205500 Mb /dev/rhdisk29
  75:     102750 Mb /dev/rhdisk30
  76:     102750 Mb /dev/rhdisk31
  77:     102750 Mb /dev/rhdisk32
  78:     102750 Mb /dev/rhdisk33
  79:     102750 Mb /dev/rhdisk34
  80:     102750 Mb /dev/rhdisk35
  81:     102750 Mb /dev/rhdisk36
  82:     102750 Mb /dev/rhdisk37
  83:     102750 Mb /dev/rhdisk38
  84:     102750 Mb /dev/rhdisk39
  85:     102750 Mb /dev/rhdisk40
  86:     102750 Mb /dev/rhdisk41
  87:     102750 Mb /dev/rhdisk42
  88:     102750 Mb /dev/rhdisk43
  89:     102750 Mb /dev/rhdisk44
  90:     102750 Mb /dev/rhdisk45
  91:     102750 Mb /dev/rhdisk46
  92:     102750 Mb /dev/rhdisk47
  93:     205500 Mb /dev/rhdisk48
  94:     205500 Mb /dev/rhdisk49
  95:     205500 Mb /dev/rhdisk50
  96:     205500 Mb /dev/rhdisk51
  97:     205500 Mb /dev/rhdisk52
  98:     205500 Mb /dev/rhdisk53
  99:     205500 Mb /dev/rhdisk54
 100:     205500 Mb /dev/rhdisk55
 101:     205500 Mb /dev/rhdisk56
 102:     205500 Mb /dev/rhdisk57
 103:     205500 Mb /dev/rhdisk58
 104:     205500 Mb /dev/rhdisk59
 105:       2048 Mb /dev/rhdisk60
 106:       2048 Mb /dev/rhdisk61
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
================================================================================
     +ASM1 /opt/oracle/products/11.2/grid
     +ASM2 /opt/oracle/products/11.2/grid
     +ASM4 /opt/oracle/products/11.2/grid
     +ASM3 /opt/oracle/products/11.2/grid
[XXXXXHOST]/dev> kfod op=groups

--Checking ORACLE_ASM_HOME


[XXXXHOST]/dev> kfod a='/dev/hdisk*' d=all
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
================================================================================
     +ASM1 /opt/oracle/products/11.2/grid
     +ASM2 /opt/oracle/products/11.2/grid
     +ASM4 /opt/oracle/products/11.2/grid
     +ASM3 /opt/oracle/products/11.2/grid


Best Regards,
Paulo Portugal

Standby Database errors in alert ORA-12801ORA-00600 ORA-10567 ORA-10564 ORA-01110 ORA-10561

If your MRP process stop on standby database with errors like below:


Errors in file /dbmid/admin/XXX/bdump/bwmdpr1_mrp0_1646.trc:
ORA-12801: error signaled in parallel query server P007, instance hostname055:XXX1 (1)
ORA-00600: internal error code, arguments: [3020], [37], [1277558], [156466806], [], [], [], []
ORA-10567: Redo is inconsistent with data block (file# 37, block# 1277558)
ORA-10564: tablespace TS_XXX_INDEX_M
ORA-01110: data file 37: '+DG_XXX_DATA/xxx/datafile/tsin_xxx_index_m.dbf'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 135950
Wed Aug 21 15:24:02 2013
MRP0: Background Media Recovery process shutdown (XXXR1)


Action Plan:

1-Check the object in primary database
select OWNER, OBJECT_NAME, SUBOBJECT_NAME, OBJECT_TYPE 
     from DBA_OBJECTS 
     where DATA_OBJECT_ID = 135950;

In My case it was an index and was not corrupted on primary database.

2-Start recover of standby allowing one corruption using this command below:

SQL> alter database recover automatic standby database allow 1 corruption;

Best Regards,
Paulo Portugal

Start and Stop GoldenGate Veridata Command Line

-> Start
cd $GG_VER_HOME/web/bin
./startup.sh

-> Stop

cd $GG_VER_HOME/web/bin
./shutdown.sh


Best Regards,
Paulo Portugal