Pages

Tuesday, September 15, 2015

RMAN常用命令

8.4 RMAN 命令知多少


  RMAN 作为一个专业的备份工具,它几乎考虑到了备份及恢复过程中可能遇到的各种操作,因此它也提供了各种各样的命令来为这些操作提供便利。

8.4.1 显示RMAN默认配置——SHOW命令

  在RMAN中有一系列的默认配置,这些配置决定了备份和恢复时的一些默认选项(此节内容枯燥,眼神不好的同学请自觉跳过),DBA可以通过 SHOW 命令来显示当前的配置参数。例如:
    RMAN>  SHOW ALL;
    RMAN configuration parameters are:
    CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
    CONFIGURE BACKUP OPTIMIZATION OFF; # default
    CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
    CONFIGURE CONTROLFILE AUTOBACKUP OFF; # default
    CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO  ' %F ' ; # default
    CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
    CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
    CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
    CONFIGURE CHANNEL 1 DEVICE TYPE DISK FORMAT   'F:\ORACLE\BACKUP\BAK_%U ' ;
    CONFIGURE MAXSETSIZE TO UNLIMITED; # default
    CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
    CONFIGURE ENCRYPTION ALGORITHM  ' AES128 ' ; # default
    CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
    CONFIGURE SNAPSHOT CONTROLFILE NAME TO  'F:\ORACLE\PRODUCT\10.2.0\DB_2\DATABASE\SNCFJSSBOOK.ORA' ; # default
    RMAN>
  这其中的每一项配置都对应了不同的功能,关于这些配置的具体功能和使用方式,后面的小节会有详细的介绍。
    提示:配置项后面加了# default的表示该项为初始配置,未被修改过。
  SHOW 命令的使用也相当灵活,其后加上不同类型的配置参数,就可以显示不同类型的配置,例如:
    RMAN>  SHOW CONTROLFILE AUTOBACKUP;
    RMAN configuration parameters are:
    CONFIGURE CONTROLFILE AUTOBACKUP OFF; # default

8.4.2   列出备份信息——LIST命令

  LIST 命令用来查看通过RMAN生成的备份集、备份镜像、归档文件等,这个命令使用也比较简单,用LIST+相应关键字即可,例如:
  • 列出数据库中所有的备份信息:
    RMAN> LIST BACKUP;
  • 列出所有备份的控制文件信息:
    RMAN> LIST BACKUP OF CONTROLFILE;
  • 列出指定数据文件的备份信息:
    RMAN> LIST BACKUP OF DATAFILE  ' F:\ORACLE\ORADATA\JSSBOOK\SCOTT_TBS01.DBF ' ;
  或
    RMAN> LIST BACKUP OF DATAFILE 5;
  注:DATAFILE序号可以通过动态性能视图 V$DATAFILE 或数据字典 DBA_DATA_FILES 中查询。
  • 列出所有备份的归档文件信息:
    RMAN> LIST BACKUP OF ARCHIVELOG ALL;
  • 列出指定表空间的备份信息:
    RMAN> LIST COPY OF TABLESPACE  ' SYSTEM ' ;
  • 列出某个设备上的所有信息:
    RMAN> LIST DEVICE TYPE DISK BACKUP;
  • 列出数据库当前所有归档:
    RMAN> LIST ARCHIVELOG ALL;
  • 列出所有无效备份:
    RMAN> LIST EXPIRED BACKUP;

8.4.3   删除备份——DELETE命令

  用于删除RMAN备份记录及相应的物理文件。当使用RMAN执行备份操作时,会在RMAN资料库(RMAN Repository)中生成RMAN备份记录,默认情况下RMAN备份记录会被存放在目标数据库的控制文件中,如果配置了恢复目录(Recovery  C atalog ),那么该备份记录也会被存放到恢复目录中。
  RMAN 中的DELETE命令就是用来删除记录(某些情况下并非删除记录,而是打上删除标记),以及这些记录关联的物理备份片段。
  • 删除过期备份。当使用RMAN命令执行备份操作时,RMAN会根据备份冗余策略确定备份是否过期。
    RMAN>  DELETE OBSOLETE;
  • 删除无效备份。首先执行 CROSSCHECK 命令核对备份集,如果发现备份无效(比如备份对应的数据文件损坏或丢失),RMAN会将该备份集标记为EXPIRED状态。要删除相应的备份记录,可以执行 DELETE EXPIRED BACKUP 命令:
    RMAN>  DELETE EXPIRED BACKUP;
  • 删除EXPIRED副本,如下所示:
    RMAN>  DELETE EXPIRED COPY;
  • 删除特定备份集,如下所示:
    RMAN>  DELETE BACKUPSET 19;
  • 删除特定备份片,如下所示:
    RMAN>  DELETE BACKUPPIECE  ' d:\backup\DEMO_19.bak ' ;
  • 删除所有备份集,如下所示:
    RMAN>  DELETE BACKUP;
  • 删除特定映像副本,如下所示:
    RMAN>  DELETE DATAFILE COPY  ' d:\backup\DEMO_19.bak ' ;
  • 删除所有映像副本,如下所示:
    RMAN>  DELETE COPY;
  • 在备份后删除输入对象,如下所示:
    RMAN>  BACKUP ARCHIVELOG ALL DELETE INPUT;
    RMAN>  DELETE BACKUPSET 22 FORMAT  =  'd:\backup\%u.bak'  DELETE INPUT;   
提示:什么是RMAN资料库(RMAN Repository)?
这是一个逻辑概念,指RMAN备份恢复需要用到的数据信息,如备份集路径、归档文件路径等。如果使用了恢复目录,则这部分数据保存于恢复目录中,否则只会保存在目标数据库的控制文件中。
=================================================


8.4.4 报表显示——REPORT命令

  REPORT 命令主要是用于当前备份信息的分析,如哪些备份已经过期,哪些文件该备份,哪些备份不可用,哪些备份可以删除之类的。
  REPORT 命令特别提供了一个SCHEMA关键字,可以用来查看数据库的模式信息(注意此模式非用户的模式哟),比如某个时间段的数据库结构,不过,使用这些功能的前提就是必须首先拥有足够的备份,不然使用REPORT命令得出的结果恐怕也满足不了你的需求啊。
  • 查看7天前数据库的模式:
    RMAN> REPORT SCHEMA AT TIME  ' SYSDATE-7 ' ;
  注意,查看模式信息必须连接到catalog数据库。
  • 查看所有需要备份的文件:
    RMAN> REPORT NEED BACKUP; 
  • 查看指定表空间是否需要备份:
    RMAN> REPORT NEED BACKUP TABLESPACE SYSTEM;
  • 查看过期备份:
    RMAN> REPORT OBSOLETE;

8.4.5 执行检查——CROSSCHECK命令

  CROSSCHECK 命令用来检查备份或归档物理文件的状态,如果物理文件存在,并且控制文件(或恢复目录)中有匹配记录,则标记该对象的状态为 AVAILABLE (有效),如果文件已经不存在了,则标记该对象状态为 EXPIRED (无效)。
  CROSSCHECK 命令并不会主动删除文件(也确实没有这个功能),它只是修改控制文件中对应记录的状态标志,因此如果想删除那些 EXPIRED 的记录,还是得手动通过 DELETE EXPIRED 命令进行。
  检查所有归档文件:
    RMAN> CROSSCHECK ARCHIVELOG ALL;
    提示:
    当操作系统环境变量NLS_LANG指定为中文SIMPLIFIED CHINESE_CHINA时,执行CROSSCHECK命令检查归档,如果结果显示“对归档日志的验证失败”,但你又确定该文件存在,别着急,这其实是Oracle 10g版本中的一个bug,验证归档文件显示的结果是反着的,如果显示验证失败说明验证是成功的,反倒是提示验证成功的话你就要小心了,说明验证失败了。 这不是CROSSCHECK命令自身的问题,而是Oracle中英文翻译的问题,设置NLS_LANG为AMERICAN_AMERICA,在英文环境下检查显示一切正常。
  检查所有备份集:
    RMAN> CROSSCHECK BACKUP;

8.4.6 修改状态——CHANGE命令

  CHANGE 命令可以用来修改备份文件或归档文件的状态,不是指物理文件,而是这些文件在控制文件(或恢复目录)中对应记录的状态,状态有两种: AVAILABLE (可用)和 UNAVAILABLE (不可用)。
  例如,修改指定备份集状态为 UNAVAILABLE :
    RMAN> CHANGE BACKUPSET  n  UNAVAILABLE;
  注意,n 为备份集序号,指定的备份集必须确实存在,不然执行会报错。
  修改指定表空间的备份集为 UNAVAILABLE :
    RMAN> CHANGE BACKUP OF TABLESPACE USERS UNAVAILABLE;
  修改指定归档文件状态为 UNAVAILABLE :
    RMAN>  CHANGE ARCHIVELOG LOGSEQ =n  UNAVAILABLE;
  提示,这里的n为归档文件序号,可以通过 LIST ARCHIVELOG ALL 或查询 V$ARCHIVED_LOG 获取。当然你也可以指定归档文件详细路径。
  如果将指定对象状态修改为可用,执行上述命令时将关键字 UNAVAILABLE 改为 AVAILABLE 即可,这里不做演示了。
  另外,与CROSSCHECK命令不同,CHANGE命令附带了 DELETE 子句,配合使用能够在修改记录状态的同时直接删除物理文件,功能那是相当彪悍,例如,删除某个归档文件:
    RMAN>  CHANGE ARCHIVELOG LOGSEQ =n  DELETE ;
  又见到n,这个n是指啥俺先不说,让你猜,猜对了俺就告诉你。

8.4.7 综述

  前面介绍了很多命令(即使介绍的几个也没有完全展开,只介绍了该命令的一些常用功能),其实RMAN中的命令远不止这几个(最最重要的BACKUP命令和RESTORE命令都没介绍哪,没错,俺是成心的),可调用参数也远不止这些。你是不是已经觉着命令太多,而且每个命令又有太多调用参数,完全记不住?老实说,俺也记不全,不过俺有超级必杀技,命令再多也不怕。悄悄告诉你,俺的秘技就是完全不用记,别倒别倒,俺不准备教你学太极,俺刚才的话还没说完。俺的意思是说,记不住不要紧,只要注意看RMAN的提示信息就好了。
  比如想查看已经备份的归档,只记得要用LIST命令查看,后面应该加什么参数全忘了,没关系,那就先执行LIST命令好了:
    RMAN> LIST;
    RMAN-01009: syntax error: found ";": expecting one of: " all, archivelog, backup, backuppiece, backupset, backed, completed, copy, controlfilecopy, datafilecopy, device, expired, global, incarnation, like, proxy, recoverable, script, tag "
  你看,返回了一堆的信息,提示你语法错误,只能支持上述加粗显示的那些关键字。
  根据提示信息,隐约记得应该是BACKUP关键字,那就敲上BACKUP试试看:
    RMAN> LIST BACKUP OF;
    RMAN-01009: syntax error: found ";": expecting one of: " archivelog, controlfile, database, datafile, spfile, tablespace "
  哈哈,又报错了啊, en ,提示越来越清楚了,加参数 ARCHIVELOG 再试试:
    RMAN> LIST BACKUP OF ARCHIVELOG ;
    RMAN-01009: syntax error: found ";": expecting one of: " all, from, high, like, logseq, low, scn, sequence, time, until "
  OK ,基本明了,我们的目的是要查看所有备份的归档,最后再加参数ALL就好了:
    RMAN> LIST BACKUP OF ARCHIVELOG ALL;
    ...
  结果显示出来了吧,目标实现,竣工!
  如果你连RMAN中有哪些命令都忘记了,没关系,随便输入个字母再按回车键试试:
    RMAN-01009: syntax error: found "identifier": expecting one of: "allocate, alter, backup, beginline, blockrecover, catalog, change, connect, copy, convert, create, crosscheck, configure, duplicate, debug, delete, drop, exit, endinline, flashback, host, {, library, list, mount, open, print, quit, recover, register, release, replace, report, renormalize, reset, restore, resync, rman, run, rpctest, set, setlimit, sql, switch, spool, startup, shutdown, send, show, test, transport, upgrade, unregister, validate"
  虽然又报出了RMAN-01009错误,但是你看,RMAN中支持的命令是不是也都列出来了呢?这个示例告诉我们,出错也不全是坏事哟。嘿嘿,自己躲墙角偷偷乐去吧,旁人俺都不告诉他的。
  不过如果次次输入都报错就显得太不专业了,因此偷偷用就好,如果你的工作时刻被领导关注着,那俺还是建议你务必要牢记一些常用命令。

Saturday, September 5, 2015

Troubleshooting DataGuard Environment

Helping in the OTN  forums – Data Guard (OTN) , They used to post without proper information which is related to Data Guard. I often face help requests without  any usefull information, regarding to a problem. And this situation happens again and again.”
So, I’d like to offer two scripts, supposed to be run at primary and standby nodes, to help OPs gather necessary information
Questions specific to DataGuard includes “Physical standby”, “Logical Standby” , “DG Broker”, “DataGuard with RAC”.
These scripts help in the case of a Physical Standby configuration and basic information which is applicable for other environment too (Logical)
Generated log files will help us to detect and troubleshoot any errors and misconfigurations as soon as possible
Run Below scripts from SYS user from Both Primary & Standby databases.
Primary Script:-
spool dg_Primary_output.log
set feedback off
set trimspool on
set line 500
set pagesize 50
column name for a30
column display_value for a30
column ID format 99
column "SRLs" format 99
column active format 99
col type format a4
column ID format 99
column "SRLs" format 99
column active format 99
col type format a4
col PROTECTION_MODE for a20
col RECOVERY_MODE for a20
col db_mode for a15
SELECT name, display_value FROM v$parameter WHERE name IN ('db_name','db_unique_name','log_archive_config','log_archive_dest_2','log_archive_dest_state_2','fal_client','fal_server','standby_file_management','standby_archive_dest','db_file_name_convert','log_file_name_convert','remote_login_passwordfile','local_listener','dg_broker_start','dg_broker_config_file1','dg_broker_config_file2','log_archive_max_processes') order by name;
col name for a10
col DATABASE_ROLE for a10
SELECT name,db_unique_name,protection_mode,DATABASE_ROLE,OPEN_MODE,switchover_status from v$database;
select thread#,max(sequence#) from v$archived_log group by thread#;
col severity for a15
col message for a70
col timestamp for a20
select severity,error_code,to_char(timestamp,'DD-MON-YYYY HH24:MI:SS') "timestamp" , message from v$dataguard_status where dest_id=2;
select ds.dest_id id
, ad.status
, ds.database_mode db_mode
, ad.archiver type
, ds.recovery_mode
, ds.protection_mode
, ds.standby_logfile_count "SRLs"
, ds.standby_logfile_active active
, ds.archived_seq#
from v$archive_dest_status ds
, v$archive_dest ad
where ds.dest_id = ad.dest_id
and ad.status != 'INACTIVE'
order by
ds.dest_id;
column FILE_TYPE format a20
col name format a60
select    name
,    floor(space_limit / 1024 / 1024) "Size MB"
,    ceil(space_used  / 1024 / 1024) "Used MB"
from    v$recovery_file_dest
order by name;
spool off
This script will generate dg_Primary_output.log file where user is connected to SQLPlus from shell.
Standby Script:-
spool dg_standby_output.log
set feedback off
set trimspool on
set line 500
set pagesize 50
set linesize 200
column name for a30
column display_value for a30
col value for a10
col PROTECTION_MODE for a15
col DATABASE_Role for a15
SELECT name, display_value FROM v$parameter WHERE name IN ('db_name','db_unique_name','log_archive_config','log_archive_dest_2','log_archive_dest_state_2','fal_client','fal_server','standby_file_management','standby_archive_dest','db_file_name_convert','log_file_name_convert','remote_login_passwordfile','local_listener','dg_broker_start','dg_broker_config_file1','dg_broker_config_file2','log_archive_max_processes') order by name;
col name for a10
col DATABASE_ROLE for a10
SELECT name,db_unique_name,protection_mode,DATABASE_ROLE,OPEN_MODE from v$database;
select thread#,max(sequence#) from v$archived_log where applied='YES' group by thread#;
select process, status,thread#,sequence# from v$managed_standby;
SELECT ARCH.THREAD# "Thread", ARCH.SEQUENCE# "Last Sequence Received", APPL.SEQUENCE# "Last Sequence Applied", (ARCH.SEQUENCE# - APPL.SEQUENCE#) "Difference"
FROM
(SELECT THREAD# ,SEQUENCE# FROM V$ARCHIVED_LOG WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) FROM V$ARCHIVED_LOG GROUP BY THREAD#)) ARCH,
(SELECT THREAD# ,SEQUENCE# FROM V$LOG_HISTORY WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) FROM V$LOG_HISTORY GROUP BY THREAD#)) APPL
WHERE ARCH.THREAD# = APPL.THREAD# ORDER BY 1;
col name for a30 
select * from v$dataguard_stats; 
select * from v$archive_gap; 
col name format a60 
select name,floor(space_limit / 1024 / 1024) "Size MB" ,ceil(space_used  / 1024 / 1024) "Used MB" from    v$recovery_file_dest order by name; 
spool off

This script will generate dg_standby_output.log file where user is connected to SQLPlus from shell. Further there would be changes in this script according to requirement. :) Any comments always welcome.

Oracle RAC Cache Fusion

Oracle Tips by Rohit GuptaAugust 12, 2011
Tips from the Trenches by Rohit Gupta
This paper is by no means an exhaustive discussion on RAC or its architecture. The focus here is only on the concept called CACHE FUSION which is core to the functioning of a RAC setup. The intention is that after reading the paper, readers should be well acquainted with CACHE FUSION and how it works to resolve the scenarios which could be contentious in a normal multi-node multi-user setup.  
Introduction  
Some logical questions in case of a RAC database would be around the shared and common components. For example, do the nodes share storage? Do the nodes share memory? How do the nodes communicate? Since this is not a RAC study guide, answering all such questions is out of scope for this article. But one question we are definitely trying to address here is the shared memory part. Though each instance has a local buffer cache, CACHE FUSION causes sharing of cache and hence resolves the problems like concurrency etc. Needless to say, the sharing is completely transparent to the end users.  
First things first - What is Cache Fusion?  
Whoever knows the basics of RAC should very well be aware of the fact that CACHE FUSION is one of the most important and interesting concepts in a RAC setup. As the name suggests, CACHE FUSION is the amalgamation of cache from each node/instance participating in the RAC, but it is not any physically secured memory component which can be configured unlike the usual buffer cache (or other SGA components) which is local to each node/instance.  
We know that every instance of the RAC database has its own local buffer cache which performs the usual cache functionality for that instance. Now there could be occasions when a transaction/user on instance A needs to access a data block which is being owned/locked by the other instance B. In such cases, the instance A will request instance B for that data block and hence accesses the block through the interconnect mechanism. This concept is known as CACHE FUSION where one instance can work on or access a data block in other instance?s cache via the high speed interconnect.  
Cache Fusion architecture helps resolve each possible type of contentions that could be thought of in a multi-node RAC setup. We will look at them in detail in coming sections but first let us understand few very important terms/concepts which will be useful in understanding the contentions which we are going to discuss in later sections.
 Global Cache Service  
Global Cache Service (GCS) is the heart of Cache Fusion concept. It is through GCS that data integrity in RAC is maintained when more than one instance need a particular data block. Instances look up to the GCS for fulfilling their data block needs.
 GCS is responsible for:
  1. Tracking the data block
  2.  Accepting the data block requests from instances
  3. Informing the holding instance to release the lock on the data block or ship a CR image
  4. Coordinating the shipping of data blocks as needed between the instance through the interconnect
  5. Informing the instances to keep or discard PIs  
More about the above functions will be clear from the following discussion on contention. Please note that GCS is available in the form of the background process called LMS.
Past Image  
The concept of Past Image is very specific to RAC setup. Consider an instance holding exclusive lock on a data block for updates. If some other instance in the RAC needs the block, the holding instance can send the block to the requesting instance (instead of writing it to disk) by keeping a PI (Past Image) of the block in its buffer cache. Basically, PI is the copy of the data block before the block is written to the disk.
  •  There can be more than one PI of the block at a time across the instances. In case there is some instance crash/failure in the RAC and a recovery is required, Oracle is able to re-construct the block using these Past Images from all the instances.
  •  When a block is written to the disk, all Past Images of that block across the instances are discarded. GCS informs all the instances to do this. At this time, the redo logs containing the redo for that data block can also be overwritten because they are no longer needed for recovery.
Consistent Read
A consistent read is needed when a particular block is being accessed/modified by transaction T1 and at the same time another transaction T2 tries to access/read the block. If T1 has not been committed, T2 needs a consistent read (consistent to the non-modified state of the database) copy of the block to move ahead. A CR copy is created using the UNDO data for that block.   A sample series of steps for a CR in a normal setup would be:
  1. Process tries to read a data block
  2. Finds an active transaction in the block
  3. Then checks the UNDO segment to see if the transaction has been committed or not
  4. If the transaction has been committed, it creates the REDO records and reads the block
  5. If the transaction has not been committed, it creates a CR block for itself using the UNDO/ROLLBACK information.
  6. Creating a CR image in RAC is a bit different and can come with some I/O overheads. This is because the UNDO could be spread across instances and hence to build a CR copy of the block, the instance might has to visit UNDO segments on other instances and hence perform certain extra I/O
Possible contentions in a RAC setup and How CACHE FUSION helps resolve them  
As mentioned above, CACHE FUSION helps resolve all the possible contentions that could happen between instances in a RAC setup.   There are 3 possible contentions in a RAC setup which we are going to discuss in detail here with a mention of cache fusion where ever applicable.
Our discussion thus far should help understand the following discussion on contentions and their resolutions better.
  1.  Read/Read contention:  Read-Read contention might not be a problem at all because the table/row will be in a shared lock mode for both transactions and none of them is trying an exclusive lock anyways.
  2.  Read/Write contention:   This one is interesting. 

    Here is more about this contention and how the concept of cache fusion helps resolve this contention
  3. A data block is in the buffer cache of instance A and is being updated. An exclusive lock has been acquired on it.
  4. After some time instance B is interested in reading that same data block and hence sends a request to GCS. So far so good ? Read/Write contention has been induced
  5. GCS checks the availability of that data block and finds that instance A has acquired an exclusive lock. Hence, GCS asks instance A to release the block for instance B.
  6. Now there are two options ? either instance A releases the lock on that block (if it no longer needs it) and lets instance B read the block from the disk OR instance A creates a CR image of the block in its own buffer cache and ships it to the requesting instance via interconnect
  7. The holding instance notifies the GCS accordingly (if the lock has been released or the CR image has been shipped)
  8. Creation of CR image, shipping it to the requesting instance and involvement of GCS is where CACHE FUSION comes into play
  1. Write/Write contention:  
This is the case where both instance A as well as B are trying to acquire an exclusive lock on the data block. A data block is in the buffer cache of instance A and is being updated. An exclusive lock has been acquired on it
  1. Instance B send the data block request to the GCS
  2. GCS checks the availability of that data block and finds that instance A has acquired an exclusive lock. Hence, GCS asks instance A to release the block for instance B
  3. There are 2 options - either instance A releases the lock on that block (if it no longer needs it) and lets instance B read the block from the disk OR instance A creates a PI image of the block in its own buffer cache, makes the redo entries and ships the block to the requesting instance via interconnect
  4. Holding instance also notifies the GCS that lock has been released and a PI has been preserved
  5. Instance B now acquires the exclusive lock on that block and continues with its normal processing. At this point GCS records that data block is now with instance B
  6. The whole mechanism of resolving this contention with the due involvement of GCS is attributed to the CACHE FUSION.
PI image VS CR image  
Let us just halt and understand some basic stuff - Wondering why CR image used in Read-Write contention and PI image used in Write-Write contention? What is the difference?  
  1.  CR image was shipped to avoid Read-Write type of contention because the requesting instance doesn?t wants to perform a write operation and hence won?t need an exclusive lock on the block. Thus for a read operation, the CR image of the block would suffice. Whereas for Write-Write contention, the requesting instance also needs to acquire an exclusive lock on the data block. So to acquire the lock for write operations, it would need the actual block and not the CR image. The holding instance hence sends the actual block but is liable to keep the PI of the block until the block has been written to the disk. So if there is any instance failure or crash, Oracle is able to build the block using the PI from across the RAC instances (there could be more than on PI of a data block before the block has actually been written to the disk). Once the block is written to the disk, it won?t need a recovery in case of a crash and hence associated PIs can be discarded.
  2.  Another difference of course is that the CR image is to be shipped to the requesting instance where as the PI has to be kept by the holding instance after shipping the actual block.  
What about UNDO?  
This discussion is not about UNDO management in RAC but here is a brief about UNDO in a RAC scenario. UNDO is generated separately on each instance just similar to a standalone database.  Each instance has its own UNDO tablespace. The UNDO data of all instances is used by holding instance to build CR image in case of contention
*******************************************************************************************
If data were never changed, life would be easier. Each node in the cluster would just read the data block from disk. Sadly, things become more difficult when we have to deal with transactions that modify data in the database. Since this in an Oracle database, we live in a world where writers never block readers, which is a very good thing. A transaction modifying a row in a table is not allowed to block another session that needs to read that row. Yet the session reading the row is not allowed to see the other transaction?s changes until those changes are committed.
In a single instance database, Oracle generates a consistent read, an image of the data block before the transaction started. Oracle uses the information in the Undo tablespace to generate the consistent read image. When Oracle 8i introduced Cache Fusion, the only block transfers across the cluster interconnect were to transfer consistent read images from the node that changed the block to the node that needed to read the block. Oracle 8i?s Cache Fusion alleviated read/write contention issues for Oracle RAC. However, write/write contention still required disk pinging. Oracle 9i?s Cache Fusion improved write/write block contention. If one node modified a block and another instance needed to modify the same block, instead of writing the dirty buffer to disk, the dirty block is transferred to the requesting node through the cluster interconnect.

In order to facilitate Cache Fusion, we still need the Buffer Cache, the Shared Pool and the Undo tablespace just like a single-instance database. However, for Oracle RAC, we need the Buffer Caches on all instances to appear to be global across the cluster.

Hence, extra coordination is needed in the cluster to make a collection of instances work together. For starters, we need a Global Resource Directory (GRD) to be able to keep track of the resources in the cluster.  There is no true concept of a master node in Oracle RAC. Instead, each instance in the cluster becomes the resource masterfor a subset of resources. The Global Cache Services (GCS) are responsible for facilitating the transfer of blocks from one instance to another. A single-instance database relies on enqueues (locks) to protect two processes from simultaneously modifying the same row. Similarly, we need enqueues in Oracle RAC but since the Buffer Cache is now global, the enqueues on the resources must be global as well.

It should be no surprise that the Global Enqueue Services (GES) is responsible for managing locks across the cluster. As a side note, GES was previously called the Distributed Lock Manager (DLM). When Oracle introduced their first cluster on DEC VAX systems, the clustered database used VAX?s cluster lock manager but it was not designed for the demands of a transactional database and did not scale well. Oracle designed the DLM to have a scalable global lock manager. The DLM still persists in many publications.

The processes running to support an Oracle RAC instance include:

?         LMS:  This process is GCS. This process used to be called the Lock Manager Server.

?         LMON:  The Lock Monitor. This process is the GES master process.
?         LMD:  The Lock Manager Daemon. This process manages incoming lock requests. 
?         LCK0: The instance enqueue process. This process manages lock requests for library cache objects.

Tuesday, August 25, 2015

How To Use Find and Locate to Search for Files on a Linux VPS

Sep 27, 2013  Linux BasicsSystem ToolsLinux Commands

Introduction

One problem users run into when first dealing with a Linux machine is how to find the files they are looking for.
This guide will cover how to use the aptly named find command. This will help you search for files on your VPS using a variety of filters and parameters. We will also briefly cover the locate command, which can be used to search for commands in a different way.

Finding by Name

The most obvious way of searching for files is by name.
To find a file by name, type:
find -name "query"
This will be case sensitive, meaning a search for "file" is different than a search for "File".
To find a file by name, but ignore the case of the query, type:
find -iname "query"
If you want to find all files that don't adhere to a specific pattern, you can invert the search with "-not" or "!". If you use "!", you must escape the character so that bash does not try to interpret it before find can act:
find -not -name "query_to_avoid"
Or
find \! -name "query_to_avoid"

Finding by Type

You can specify the type of files you want to find with the "-type" parameter. It works like this:
find -type type_descriptor query
Some of the most common descriptors that you can use to specify the type of file are here:
  • f: regular file
  • d: directory
  • l: symbolic link
  • c: character devices
  • b: block devices
For instance, if we wanted to find all of the character devices on our system, we could issue this command:
find / -type c
/dev/parport0
/dev/snd/seq
/dev/snd/timer
/dev/autofs
/dev/cpu/microcode
/dev/vcsa7
/dev/vcs7
/dev/vcsa6
/dev/vcs6
/dev/vcsa5
/dev/vcs5
/dev/vcsa4
. . .
We can search for all files that end in ".conf" like this:
find / -type f -name "*.conf"
/var/lib/ucf/cache/:etc:rsyslog.d:50-default.conf
/usr/share/base-files/nsswitch.conf
/usr/share/initramfs-tools/event-driven/upstart-jobs/mountall.conf
/usr/share/rsyslog/50-default.conf
/usr/share/adduser/adduser.conf
/usr/share/davfs2/davfs2.conf
/usr/share/debconf/debconf.conf
/usr/share/doc/apt-utils/examples/apt-ftparchive.conf
. . .

Filtering by Time and Size

Find gives you a variety of ways to filter results by size and time.

Size

You can filter by size with the use of the "-size" parameter.
We add a suffix on the end of our value that specifies how we are counting. These are some popular options:
  • c: bytes
  • k: Kilobytes
  • M: Megabytes
  • G: Gigabytes
  • b: 512-byte blocks
To find all files that are exactly 50 bytes, type:
find / -size 50c
To find all files less than 50 bytes, we can use this form instead:
find / -size -50c
To Find all files more than 700 Megabytes, we can use this command:
find / -size +700M

Time

Linux stores time data about access times, modification times, and change times.
  • Access Time: Last time a file was read or written to.
  • Modification Time: Last time the contents of the file were modified.
  • Change Time: Last time the file's inode meta-data was changed.
We can use these with the "-atime", "-mtime", and "-ctime" parameters. These can use the plus and minus symbols to specify greater than or less than, like we did with size.
The value of this parameter specifies how many days ago you'd like to search.
To find files that have a modification time of a day ago, type:
find / -mtime 1
If we want files that were accessed in less than a day ago, we can type:
find / -atime -1
To get files that last had their meta information changed more than 3 days ago, type:
find / -ctime +3
There are also some companion parameters we can use to specify minutes instead of days:
find / -mmin -1
This will give the files that have been modified type the system in the last minute.
Find can also do comparisons against a reference file and return those that are newer:
find / -newer myfile

Finding by Owner and Permissions

You can also search for files by the file owner or group owner.
You do this by using the "-user" and "-group" parameters respectively. Find a file that is owned by the "syslog" user by entering:
find / -user syslog
Similarly, we can specify files owned by the "shadow" group by typing:
find / -group shadow
We can also search for files with specific permissions.
If we want to match an exact set of permissions, we use this form:
find / -perm 644
This will match files with exactly the permissions specified.
If we want to specify anything with at least those permissions, you can use this form:
find / -perm -644
This will match any files that have additional permissions. A file with permissions of "744" would be matched in this instance.

Filtering by Depth

For this section, we will create a directory structure in a temporary directory. It will contain three levels of directories, with ten directories at the first level. Each directory (including the temp directory) will contain ten files and ten subdirectories.
Make this structure by issuing the following commands:
cd
mkdir -p ~/test/level1dir{1..10}/level2dir{1..10}/level3dir{1..10}
touch ~/test/{file{1..10},level1dir{1..10}/{file{1..10},level2dir{1..10}/{file{1..10},level3dir{1..10}/file{1..10}}}}
cd ~/test
Feel free to check out the directory structures with ls and cd to get a handle on how things are organized. When you are finished, return to the test directory:
cd ~/test
We will work on how to return specific files from this structure. Let's try an example with just a regular name search first, for comparison:
find -name file1
./level1dir7/level2dir8/level3dir9/file1
./level1dir7/level2dir8/level3dir3/file1
./level1dir7/level2dir8/level3dir4/file1
./level1dir7/level2dir8/level3dir1/file1
./level1dir7/level2dir8/level3dir8/file1
./level1dir7/level2dir8/level3dir7/file1
./level1dir7/level2dir8/level3dir2/file1
./level1dir7/level2dir8/level3dir6/file1
./level1dir7/level2dir8/level3dir5/file1
./level1dir7/level2dir8/file1
. . .
There are a lot of results. If we pipe the output into a counter, we can see that there are 1111 total results:
find -name file1 | wc -l
1111
This is probably too many results to be useful to you in most circumstances. Let's try to narrow it down.
You can specify the maximum depth of the search under the top-level search directory:
find -maxdepth num -name query
To find "file1" only in the "level1" directories and above, you can specify a max depth of 2 (1 for the top-level directory, and 1 for the level1 directories):
find -maxdepth 2 -name file1
./level1dir7/file1
./level1dir1/file1
./level1dir3/file1
./level1dir8/file1
./level1dir6/file1
./file1
./level1dir2/file1
./level1dir9/file1
./level1dir4/file1
./level1dir5/file1
./level1dir10/file1
That is a much more manageable list.
You can also specify a minimum directory if you know that all of the files exist past a certain point under the current directory:
find -mindepth num -name query
We can use this to find only the files at the end of the directory branches:
find -mindepth 4 -name file
./level1dir7/level2dir8/level3dir9/file1
./level1dir7/level2dir8/level3dir3/file1
./level1dir7/level2dir8/level3dir4/file1
./level1dir7/level2dir8/level3dir1/file1
./level1dir7/level2dir8/level3dir8/file1
./level1dir7/level2dir8/level3dir7/file1
./level1dir7/level2dir8/level3dir2/file1
. . .
Again, because of our branching directory structure, this will return a large number of results (1000).
You can combine the min and max depth parameters to focus in on a narrow range:
find -mindepth 2 -maxdepth 3 -name file
./level1dir7/level2dir8/file1
./level1dir7/level2dir5/file1
./level1dir7/level2dir7/file1
./level1dir7/level2dir2/file1
./level1dir7/level2dir10/file1
./level1dir7/level2dir6/file1
./level1dir7/level2dir3/file1
./level1dir7/level2dir4/file1
./level1dir7/file1
. . .

Executing and Combining Find Commands

You can execute an arbitrary helper command on everything that find matches by using the "-exec" parameter. This is called like this:
find find_parameters -exec command_and_params {} \;
The "{}" is used as a placeholder for the files that find matches. The "\;" is used so that find knows where the command ends.
For instance, we could find the files in the previous section that had "644" permissions and modify them to have "664" permissions:
cd ~/test
find . -type f -perm 644 -exec chmod 664 {} \;
We could then change the directory permissions like this:
find . -type d -perm 755 -exec chmod 700 {} \;
If you want to chain different results together, you can use the "-and" or "-or" commands. The "-and" is assumed if omitted.
find . -name file1 -or -name file9 

Find Files Using Locate

An alternative to using find is the locate command. This command is often quicker and can search the entire file system with ease.
You can install the command with apt-get:
sudo apt-get update
sudo apt-get install mlocate
The reason locate is faster than find is because it relies on a database of the files on the filesystem.
The database is usually updated once a day with a cron script, but you can update it manually by typing:
sudo updatedb
Run this command now. Remember, the database must always be up-to-date if you want to find recently acquired or created files.
To find files with locate, simply use this syntax:
locate query
You can filter the output in some ways.
For instance, to only return files containing the query itself, instead of returning every file that has the query in the directories leading to it, you can use the "-b" for only searching the "basename":
locate -b query
To have locate only return results that still exist in the filesystem (that were not remove between the last "updatedb" call and the current "locate" call), use the "-e" flag:
locate -e query
To see statistics about the information that locate has cataloged, use the "-S" option:
locate -S
Database /var/lib/mlocate/mlocate.db:
    3,315 directories
    37,228 files
    1,504,439 bytes in file names
    594,851 bytes used to store database

Conclusion

Both find and locate are good ways to find files on your system. It is up to you to decide which of these tools is appropriate in each situation.
Find and locate are powerful commands that can be strengthened by combining them with other utilities through pipelines. Experiment with filtering by using commands like wcsort, and grep.
By Justin Ellingwood