[Solr实践]自定义SolrEventListener实现searcher的autowarm策略

Posted in solr on 九月 5th, 2010 by kafka0102

Solr的searcher autowarm(预热)有两个时机,一个是系统启动时(firstSearcher),一个是使用新的searcher替换旧的searcher时(newSearcher)。Solr支持在solrconfig.xml中对SolrCore配置SolrEventListener来实现自定义的autowarm。通常来说,Solr提供的默认实现QuerySenderListener就够用了。在我的需求中,希望solrconfig.xml中配置的SolrEventListener是针对多个SolrCore的,这要是因为我的多个SolrCore共用了一个solrconfig.xml配置。就配置autowarm的查询query来说,简单的就是配置一个常见的query,但如果系统有排序查询(sort),可以配置适宜的sort条件以预热lucene的fieldCache。下面是我自定义的SolrEventListener,效果是,如果SolrCore没有配置query,就使用default的,否则使用自己的。
实现代码修改自Solr的QuerySenderListener,代码如下:

public class TubaQuerySenderListener implements SolrEventListener {
 
  private static final Logger logger = LoggerFactory
  .getLogger(TubaQuerySenderListener.class);
 
  protected final SolrCore core;
  protected List<NamedList> queryArgs;
 
  public void init(final NamedList args) {
    final String coreName = core.getName();
    queryArgs = (List<NamedList>)args.get(coreName);
    if (queryArgs == null) {
      queryArgs = (List<NamedList>)args.get("default");
      if (queryArgs == null) {
        logger.warn("TubaQuerySenderListener not valid for core:"+coreName);
      }
    }
    logger.info("core["+coreName+"]register TubaQuerySenderListener : " + queryArgs);
  }
 
  public TubaQuerySenderListener(final SolrCore core) {
    this.core = core;
  }
 
  /**
   * Add the {@link org.apache.solr.common.params.EventParams#EVENT} with either the {@link org.apache.solr.common.params.EventParams#NEW_SEARCHER}
   * or {@link org.apache.solr.common.params.EventParams#FIRST_SEARCHER} values depending on the value of currentSearcher.
   * <p/>
   * Makes a copy of NamedList and then adds the parameters.
   *
   *
   * @param currentSearcher If null, add FIRST_SEARCHER, otherwise NEW_SEARCHER
   * @param nlst The named list to add the EVENT value to
   */
  protected NamedList addEventParms(final SolrIndexSearcher currentSearcher, final NamedList nlst) {
    final NamedList result = new NamedList();
    result.addAll(nlst);
    if (currentSearcher != null) {
      result.add(EventParams.EVENT, EventParams.NEW_SEARCHER);
    } else {
      result.add(EventParams.EVENT, EventParams.FIRST_SEARCHER);
    }
    return result;
  }
 
  @Override
  public void newSearcher(final SolrIndexSearcher newSearcher, final SolrIndexSearcher currentSearcher) {
    if (queryArgs == null) {
      return;
    }
    final SolrIndexSearcher searcher = newSearcher;
    for (final NamedList nlst : queryArgs) {
      try {
        // bind the request to a particular searcher (the newSearcher)
        final NamedList params = addEventParms(currentSearcher, nlst);
        final LocalSolrQueryRequest req = new LocalSolrQueryRequest(core,params) {
          @Override public SolrIndexSearcher getSearcher() { return searcher; }
          @Override public void close() { }
        };
 
        final SolrQueryResponse rsp = new SolrQueryResponse();
        core.execute(core.getRequestHandler(req.getParams().get(CommonParams.QT)), req, rsp);
        // Retrieve the Document instances (not just the ids) to warm
        // the OS disk cache, and any Solr document cache.  Only the top
        // level values in the NamedList are checked for DocLists.
        final NamedList values = rsp.getValues();
        for (int i=0; i<values.size(); i++) {
          final Object o = values.getVal(i);
          if (o instanceof DocList) {
            final DocList docs = (DocList)o;
            for (final DocIterator iter = docs.iterator(); iter.hasNext();) {
              newSearcher.doc(iter.nextDoc());
            }
          }
        }
        req.close();
      } catch (final Exception e) {
        // do nothing... we want to continue with the other requests.
        // the failure should have already been logged.
        logger.warn("",e);
      }
    }
    logger.info("core["+core.getName()+"]TubaQuerySenderListener newSearcher done.");
  }
 
  @Override
  public void postCommit() {
    throw new UnsupportedOperationException();
  }
 
}

solrconfig.xml中的配置示例如下,其中firstSearcher和newSearcher的配置是一样的,不过这不意味着它们是必须一样的。

	<query>
		<listener event="firstSearcher"
			class="com.tintintech.tuba.search.TubaQuerySenderListener">
			<arr name="default">
				<lst>
					<str name="q">手机</str>
					<str name="start">0</str>
					<str name="rows">10</str>
				</lst>
			</arr>
			<arr name="core1">
				<lst>
					<str name="q">手机</str>
					<str name="start">0</str>
					<str name="rows">10</str>
					<str name="sort">at desc</str>
				</lst>
			</arr>
		</listener>
 
		<listener event="newSearcher"
			class="com.tintintech.tuba.search.TubaQuerySenderListener">
			<arr name="default">
				<lst>
					<str name="q">手机</str>
					<str name="start">0</str>
					<str name="rows">10</str>
				</lst>
			</arr>
			<arr name="core1">
				<lst>
					<str name="q">手机</str>
					<str name="start">0</str>
					<str name="rows">10</str>
					<str name="sort">at desc</str>
				</lst>
			</arr>
		</listener>
	</query>

=============================== 华丽的终止符 ================================

本文作者:kafka0102,转载文章请注明来源,谢谢!!
本文链接:http://www.kafka0102.com/2010/09/326.html


相关日志


留下评论

说明:评论需要审核通过才能显示