Knowledge Base : Apache Solr 4.6 and Zookeeper 3.4.5

Issue:   Zookeeper Leader Conflict

Exception: SEVERE: ClusterState says we are the leader, but locally we don't think so*"

Cause: Zookeeper session on the leader node expires sooner than expected and its goes haywire. After this happens, leader(001) is going into 'recovery' mode and all the index updates are failing with "503- service unavailable" error message.

Resolution:

Increase the Zookeeper session timeouts in solr.xml.
Recycled Node 2 and Node 3 in sequence.
sudo service tomcat stop
sudo service tomcat start

-----------------------------------------------------------------------------------------------------------------------

Issue:  Zookeeper autopurge is not working

Exception:  java.io.FileNotFoundException: ./zookeeper.log (Permission denied)

Cause: Following paths in/opt/zookeeper/conf/zoo.cfg are incorrectly configured

dataDir=/opt/zookeeper/data
dataLogDir=/opt/zookeeper/data

Resolution:

1 .Stop Zk on all nodes. Edit  /opt/zookeeper/conf/zoo.cfg

http://zk1.deveat.pmi.org:8080/exhibitor/v1/ui/index.html

2. Update directly from ZK UI and Commit

• dataDir=/var/lib/zookeeper/data
• dataLogDir=/var/lib/zookeeper/logs
• autopurge.snapRetainCount=3
• autopurge.purgeInterval=1

3.Run this on command line to give permissions

• sudo chown -R tomcat /var/lib/zookeeper/data/myid
• sudo chown -R tomcat /var/lib/zookeeper/logs/zookeeper.log

4. Restart tomcat on all nodes

References used:

http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#Ongoing+Data+Directory+Cleanup

New in 3.4.0: When enabled, ZooKeeper auto purge feature retains the autopurge.snapRetainCount most recent snapshots and the corresponding transaction logs in the dataDir and dataLogDirrespectively and deletes the rest. Defaults to 3. Minimum value is 3.

autopurge.purgeInterval

(No Java system property)

New in 3.4.0: The time interval in hours for which the purge task has to be triggered. Set to a positive integer (1 and above) to enable the auto purging. Defaults to 0.

-------------------------------------------------------------------------------------------------------------------------