Every 20 min an Event 1069 and 1558?

DISCLAIMER: Event 1069 is an generic event therefore you should check and decide by your own if this blog post is an possible solution for your scenario and environment. This can be verified in the cluster logs when you identify "‘Failed to create cluster directory on witness”! which is generated by Quorum Agent.

Problem:

In a customer scenario we had identified every 20 min the events 1069 and 1558. They had pointed to quorum disk issues which we could not confirm in the 1st step as the disk was online and could be failed over to other nodes without any issues.

Event ID 1069 — Clustered Service or Application Availability
http://technet.microsoft.com/en-us/library/cc756225(WS.10).aspx

Event ID 1558 — Cluster Witness Functionality
http://technet.microsoft.com/en-us/library/dd353960(WS.10).aspx

After digging deeper and doing an cluster log analyses I had found an very interesting pointer there:

ERR mscs::QuorumAgent::PostOnline: ERROR_PATH_NOT_FOUND(3)’ because of ‘Failed to create cluster directory on witness, path \\?\Volume{5c65f7b0-15e4-11e0-b316-002655db949a}\Cluster

This tells me, that the cluster services has issues when trying to access the quorum disk when “he” want to write his cluster hive (=configuration) to the quorum disk.

As the cluster hive is “redundantly” available on each in the cluster and can also be manually created anytime when changing the cluster quorum configuration in your cluster, I used this scenario for troubleshooting my issue here.

Solution:

1. Changing the Quorum Modell temporary to “Node Majority” so that I can remove the Quorum Disk “Q:” from the cluster (Note: When changing quorum model, be aware of the available “votes” (keep majority) in your cluster)

image

image

2. Remove Quorum Disk from Cluster , Re-Format with NTFS and back to Cluster

3. Restore Quorum model – in my case “Node und Disk Majority” and point to new-formatted Quorum disk Q:

image

Result:: Die Cluster Hive is newly created on Quorum Disk Q:\

image

After the cluster hive is successfully created from cluster service, all entries in cluster logs and also events 1069 and 1558 are gone Winking smile

Information:

General Information’s around Cluster Logs can be found here:

How to create the cluster.log in Windows Server 2008 Failover Clustering
http://blogs.msdn.com/b/clustering/archive/2008/09/24/8962934.aspx

Troubleshooting Cluster Logs 101 – Why did the resources failover to the other node?
http://blogs.technet.com/b/askcore/archive/2008/02/06/troubleshooting-cluster-logs-101-why-did-the-resources-failover-to-the-other-node.aspx

Introduction to Cluster Diagnostics and Verification Tool for Exchange Administrators
http://technet.microsoft.com/en-us/library/aa996161(EXCHG.65).aspx

….wish you good luck with “troubleshooting” clip_image010

Best Regards

Ramazan

Advertisements

Tags: , , ,

3 Responses to “Every 20 min an Event 1069 and 1558?”

  1. joelcipriano@hotmail.com Says:

    Very nice. Exactly the explaination and procdure I went with. I lost my quorum during a SAN failure and when the smoke cleared a few months later…we had a Quorum disk but noithing in it.

    Thanks.

    Joel

  2. bg2001 Says:

    worked for me! thanks for the good info.

  3. Berner Says:

    Thanks a lot. Worked like a charm!

Any further thoughts? Let me know here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: