Monday, August 11, 2008

PRKP-1029, CRS-0211 on srvctl modify/add service

What I was trying to do:
My requirement was to change the preferred node and available node for a service.
Issues:
srvctl modify service -d ORCL -s BULK -i node3,node4 -a node1,node2
PRKP-1029 : Failed to register the service ORCL_BULK_svc.
CRS-0211: Resource 'ora.ORCL.ORCL_BULK_svc.cs' has already been reg
istered.

I tried the above command a couple of time, but gave the same error. I decided to stop the service and then try.
I decided to stop the service:
srvctl stop service -d ORCL -s BULK.
The service stopped without any issue.

But even after stopping the service, it gave the same error.

The I decided to remove the service and recreate it with the required configuration.
srvctl remove service -d ORCL -s BULK
Service got removed without any issue.

Now tried to add the service:
srvctl add service -d ORCL -s BULK -r node3,node4 -a node1,node2
PRKP-1029 : Failed to register the service ORCL_BULK_svc.
CRS-0211: Resource 'ora.ORCL.ORCL_BULK_svc.cs' has already been registered.

BAAAM, the error was not supposed to come, as I had already removed the service. Interestingly, I checked the DBA_SERVICES and still saw the entry of this service there. And also crs_stat showed this service.
Now I was in a soup as one of the most important application could not connect. Check with metalink/web the only thing I found was to "contact support". Turned to a couple of colleagues and got the suggestion to use crs_unregister. This is not a recommended thing to do, but I was in dire straits.

SO this is what we did:
crs_unregister ora.ORCL.ORCL_BULK_svc.cs

srvctl add service -d ORCL -s BULK -r node3,node4 -a node1,node2
PRKP-1029 : Failed to register the service ORCL_BULK_svc.
CRS-0211: Resource 'ora.ORCL.ORCL_BULK_svc.ORCL.srv' has already been registered.


crs_unregister ora.ORCL.ORCL_BULK_svc.ORCL.srv

srvctl add service -d ORCL -s BULK -r node3,node4 -a node1,node2
PRKP-1029 : Failed to register the service ORCL_BULK_svc.
CRS-0211: Resource 'ora.ORCL.ORCL_BULK_svc.cs' has already been registered.

We had already unregistered this one, but it gave the error for the same thing again. We ran the unregister once again:

crs_unregister ora.ORCL.ORCL_BULK_svc.cs

Looks like the sequence is to first remove the .srv and then the .cs:

crs_unregister ora.ORCL.ORCL_BULK_svc.ORCL.srv
CRS-0210: Could not find resource 'ora.ORCL.ORCL_BULK_svc.ORCL.srv'.

crs_unregister ora.ORCL.ORCL_BULK_svc.cs
CRS-0210: Could not find resource 'ora.ORCL.ORCL_BULK_svc.cs'.

Finally:
srvctl add service -d ORCL -s BULK -r node3,node4 -a node1,node2
and now it worked. And I started the service.

Details on crs_unregister from Oracle:

The crs_unregister command removes the registration information of Oracle Clusterware resources from the binary Oracle Clusterware registry database. The Oracle Clusterware will no longer acknowledge this resource. An application associated with a resource that is unregistered is no longer highly available. You must have full administrative privileges to use this command.

Upon successful completion of the crs_unregister command, the resource is removed from the online Oracle Clusterware environment. You cannot unregister a resource that is a required resource for another resource. You must stop the resource by using the crs_stop command before unregistering it.
Syntax and Options for crs_unregister

Use the crs_unregister command with the following syntax:

crs_unregister resource_name [...] [-q]

The only option available for this command is -q, that runs the crs_unregister command in quiet mode, which means no messages are displayed.
Example of crs_unregister

The following example unregisters a highly available application called postman:

crs_unregister postman

Errors for crs_unregister

Oracle displays a corresponding text message for the following error conditions:

* No root privilege
* CAA daemon is not running
* The application is running
* The application is not registered