Recovering from failed vxevac

Ilovebunt3-214x300

If you have ever evacuated disks in Veritas, every so often this will happen to hang.  Usually you terminate your session or who knows what.  Kinda like Joe Girardi's willingness to sacrifice outs for no good reason every time the Yankees hottest hitter is at the plate.  It happens, you can't explain it, you move on.  Back to technology – vxtask list shows no tasks, but you get errors trying to rerun the failed evac. 

 

For example:

Plex %5 in volume rman is locked by another utility

Plex rman-01 in volume rman is locked by another utility

Subdisk rman_7_tmp-01 in plex rman-01 is locked by another utility

vxprint -hf is our best friend, as it shows you any flags that are set

v  rman    fsgen        ENABLED  15625864960 -     ACTIVE   ATT1    –

pl %5           rman   ENABLED  11719399168 -     TEMPRM   SDMVTMP –

sd rman_6-01 %5         ENABLED  1953232896 9766166272 -    SDMVDST –

pl rman-01 rman    ENABLED  15625864960 -     ACTIVE   SDMV1   –

sd rman_1-01 rman-01 ENABLED 1953234688 0     -        -       –

sd rman_2-01 rman-01 ENABLED 1953232896 1953234688 -   -       –

sd rman_3-01 rman-01 ENABLED 1953232896 3906467584 -   -       –

sd rman_4-01 rman-01 ENABLED 1953232896 5859700480 -   -       –

sd rman_5-01 rman-01 ENABLED 1953232896 7812933376 -   -       –

sd rman_6_tmp-01 rman-01 ENABLED 1953232896 9766166272 – SDMVSRC –

sd rman_7_tmp-01 rman-01 ENABLED 1953232896 11719399168 – -    –

sd rman_8-01 rman-01 ENABLED 1953232896 13672632064 -  -       -

We can see that we have flags set on the temporary plex (from the failed evac), the subdisk for the temporary plex, the main plex, the subdisk in the main plex, as well as the volume itself.  We need to clear flags to be able to finish re-start our evac.  I will also cut the lines on the vxprint that don't change for the purpose of shortening this post.

vxmend -g rman_dg clear all rman %5

So we cleared the volume and temp plex flags, here's the vxprint -htf output afterwards

v  rman    fsgen        ENABLED  15625864960 -     ACTIVE   -       –

pl %5           rman    ENABLED  11719399168 -     TEMPRM   -       –

sd rman_6-01 %5         ENABLED  1953232896 9766166272 -    SDMVDST –

pl rman-01 rman    ENABLED  15625864960 -     ACTIVE   SDMV1   –

sd rman_6_tmp-01 rman-01 ENABLED 1953232896 9766166272 – SDMVSRC –

 

So now with the flags cleared we can remove the temporary plex

vxplex -g rman_dg -o rm dis %5

 

And once again our new vxprint -htf

v  rman    fsgen        ENABLED  15625864960 -     ACTIVE   -       –

pl rman-01 rman    ENABLED  15625864960 -     ACTIVE   SDMV1   –

sd rman_6_tmp-01 rman-01 ENABLED 1953232896 9766166272 – SDMVSRC –

 

Great, now down to two flags, the one on the plex and the one on the source disk of our original evac.  Clearing flags from subdisks is a lot trickier than clearing flags from volumes and plexes.  Because the tutil0 flga is already set, we will need to force the clear.  We clear by setting it to "".

vxedit -g rman_dg -f set tutil0="" rman_6_tmp-01

 

Once again, vxprint -htf

v  rman    fsgen        ENABLED  15625864960 -     ACTIVE   -       -

pl rman-01 rman    ENABLED  15625864960 -     ACTIVE   SDMV1   –

sd rman_6_tmp-01 rman-01 ENABLED 1953232896 9766166272 – -     –

 

And lastly, we clear the flag on the plex.  Why in this order?   Because I'm writing this up after I fixed my issues.  In the interest of not editing vxprint outputs, it's like this.  In retrospect, this could have been cleared with the first one we ran in the beginning.

vxmend -g rman_dg clear all rman rman-01

 

And finally, the way a vxprint -htf should look when all is healthy.

v  rman    fsgen        ENABLED  15625864960 -     ACTIVE   -       –

pl rman-01 rman    ENABLED  15625864960 -     ACTIVE   -       –

sd rman_1-01 rman-01 ENABLED 1953234688 0     -        -       –

sd rman_2-01 rman-01 ENABLED 1953232896 1953234688 -   -       –

sd rman_3-01 rman-01 ENABLED 1953232896 3906467584 -   -       –

sd rman_4-01 rman-01 ENABLED 1953232896 5859700480 -   -       –

sd rman_5-01 rman-01 ENABLED 1953232896 7812933376 -   -       –

sd rman_6_tmp-01 rman-01 ENABLED 1953232896 9766166272 – -     –

sd rman_7_tmp-01 rman-01 ENABLED 1953232896 11719399168 – -    –

sd rman_8-01 rman-01 ENABLED 1953232896 13672632064 -  -       –

 

At this point, feel free to proceed with your evac again.  If you're wondering what the putil and tutil fields are, here is what I found courtesy of Symantec:

http://www.symantec.com/business/support/index?page=content&id=TECH15609

 

Guest Authored By: @momkvi

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s