18 April 2019

See this article if you haven’t already set up Zookeeper for SolrCloud. Liferay also has documentation for setting up Solr and SolrCloud. I found this documentation good for the former and not so good for the latter. Starting at step 4 I found their way confusing and chose a different path. I will place the first three steps in this article as well in case the original content is moved/removed.

First you want to follow Liferay instructions to setup a standalone Solr server. If you have never setup Solr before this is recommended. It gives you a feel for how Solr should work. The key takeaways is you create a configuration in Solr for liferay by copying the default Solr configuration and overwriting some of its files with files from the Liferay Connector to Solr 7 package. The package can be found on the Liferay Marketplace.

  1. Stop the Solr server if it’s running.

  2. Navigate to the Solr_Home/configsets folder and create a folder called liferay_configs

  3. Copy the conf folder from Solr_Home/liferay to the liferay_configs folder you just created. The configset/liferay_configs folder contains the SolrCloud Liferay Portal collection configuration and is uploaded to ZooKeeper. By copying the conf folder from the liferay server configured earlier, you’re using the schema.xml and solrconfig.xml files provided with the Liferay Solr Adapter.

Next we want to edit the solr.in.sh file and add all of our Zookeeper hosts.

/your/directory/path/solr/bin/solr.in.sh
= Stuff omitted here
=  Set the ZooKeeper connection string if using an external ZooKeeper ensemble
 52 # e.g. host1:2181,host2:2181/chroot
 53 # Leave empty if not using SolrCloud
 54 ZK_HOST="192.168.0.1:9983,192.168.0.2:9983:9983/solr"
= Stuff omitted here

The "bin/solr" command line program will be used in the next steps. Documentation can be found in the Solr 7.7 reference.

We need to create our Zookeeper chroot which we will define as "/solr". This should allow us to use zookeeper for other purposes. The end of line 54, of the previous code snippet, is referencing the chroot. A znode is created that acts as a root directory for our Solr information that will keep it seperate from other applications that use the same instance of Zookeeper.

/your/directory/path/solr/bin
  $ ./solr zk mkroot /solr -z 192.168.0.1:9983,192.168.0.2:9983
  Creating Zookeeper path /solr on ZooKeeper at 192.168.0.1:9983,192.168.0.2:9983

Now we run our command to load the configset as a collection we made when we followed the Liferay directions. We also need to determine the number of shards and replicas we need. Shards are logical seperation of document storage. If you have two shards then a document will only be stored in one of them. Replicas are copies of all the documents stored. If you have one shard and two replicas then there will be one logical storage area and two copies. For our deployments we want one replica for each Solr instance. We have two Solr instances. We then specify two replicas usin the "-rf 2" option as shown below.

First start solr in cloud mode so we can make the configuration. Then run the create command to configure the core, shards and replicas.

/your/directory/path/solr/bin
  $ ./solr start -c
  Waiting up to 180 seconds to see Solr running on port 8983 [\]
  Started Solr server on port 8983 (pid=119501). Happy searching!
  $ ./solr create -c liferay -d liferay_configs -s 1 -rf 2
  Created collection 'liferay' with 1 shard(s), 2 replica(s) with config-set 'liferay'

This should create one configuration for each replica you specified. Directories will look close to the below.

/your/directory/path/solr/server/solr
4096 Aug 22 07:55 .
4096 Aug 22 07:48 ..
4096 Aug 22 07:26 configsets
4096 Aug 22 07:58 liferay_shard1_replica_n1
4096 Aug 22 07:58 liferay_shard1_replica_n2
3018 Aug 22 07:26 README.txt
2180 Aug 22 07:26 solr.xml
 975 Aug 22 07:26 zoo.cfg

Finally we want to move liferay_shard1_replica_n2 directory to our second instance and start both Solr servers in cloud mode.

As I was working through moving from Solr 4 to Solr 7 I encountered errors (below) concerning coreNodeName. This seems to concern replicas and how they were handled in early versions of Solr. There is a reference to this change, but it contains little information. From what little information was available it seems the legacyCloud being set to true obfuscates the number of replicas in your deployment.

In my situation I needed to deploy Solr in sets of 2 or 4 depending on the environment. The number of documents is small, so only one shard. I did want redundancy and responsiveness. Which means I should have one replica for each Solr instance I plan to deploy.

/your/directory/path/solr/server/logs/solr.log
2019-04-17 17:38:18.678 ERROR (coreLoadExecutor-13-thread-1-processing-n:192.168.251.225:8983_solr) [c:liferay   x:liferay] o.a.s.c     .CoreContainer Could not repair coreNodeName in core.properties file for core liferay
  2019-04-17 17:38:18.678 ERROR (coreLoadExecutor-13-thread-1-processing-n:192.168.251.225:8983_solr) [c:liferay   x:liferay] o.a.s.c     .ZkController
  org.apache.solr.common.SolrException: No coreNodeName for CoreDescriptor[name=liferay;instanceDir=/apps01/solr/server/solr/liferay]
    at org.apache.solr.cloud.ZkController.checkStateInZk(ZkController.java:1829) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e77533     9885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]
    at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1767) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e77533988     5cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]
    at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1171) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f     5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]
    at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:690) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e77533     9885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]
    at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) ~[metrics-cor     e-3.2.6.jar:3.2.6]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_201]
    at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[solr-solrj-7.7.1     .jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]

Though it is not recommended you can work around this problem with the command below. After reading Major Changes in Solr 7. This shouldn’t be necessary if you followed the directions at the beginning of this page.

/your/directory/path/solr/server/scripts/cloud-scripts
$ ./zkCli.sh -zkhost localhost:9983 -cmd clusterprop -name legacyCloud -val true
Connecting to localhost:2181
2019-07-25 09:45:01,884 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT
2019-07-25 09:45:01,888 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=acdevpoxxb5l703.usarec.army.mil
2019-07-25 09:45:01,888 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_212
2019-07-25 09:45:01,891 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2019-07-25 09:45:01,891 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre
2019-07-25 09:45:01,891 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/apps01/zookeeper/bin/../zookeeper-server/target/classes:/apps01/zookeeper/bin/../build/classes:/apps01/zookeeper/bin/../zookeeper-server/target/lib/*.jar:/apps01/zookeeper/bin/../build/lib/*.jar:/apps01/zookeeper/bin/../lib/slf4j-log4j12-1.7.25.jar:/apps01/zookeeper/bin/../lib/slf4j-api-1.7.25.jar:/apps01/zookeeper/bin/../lib/netty-3.10.6.Final.jar:/apps01/zookeeper/bin/../lib/log4j-1.2.17.jar:/apps01/zookeeper/bin/../lib/jline-0.9.94.jar:/apps01/zookeeper/bin/../lib/audience-annotations-0.5.0.jar:/apps01/zookeeper/bin/../zookeeper-3.4.14.jar:/apps01/zookeeper/bin/../zookeeper-server/src/main/resources/lib/*.jar:/apps01/zookeeper/bin/../conf:
2019-07-25 09:45:01,891 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.10.0-957.21.3.el7.x86_64
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=liferay
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/export/home/liferay
2019-07-25 09:45:01,892 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/apps01/zookeeper/bin
2019-07-25 09:45:01,893 [myid:] - INFO  [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89
ZooKeeper -server host:port cmd args
	stat path [watch]
	set path data [version]
	ls path [watch]
	delquota [-n|-b] path
	ls2 path [watch]
	setAcl path acl
	setquota -n|-b val path
	history
	redo cmdno
	printwatches on|off
	delete path [version]
	sync path
	listquota path
	rmr path
	get path [watch]
	create [-s] [-e] path data acl
	addauth scheme auth
	quit
	getAcl path
	close
	connect host:port
2019-07-25 09:45:01,926 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)

Less Is More ~ Older posts are available in the archive.