[Solr实践]自定义SolrEventListener实现searcher的autowarm策略
Posted in solr on 九月 5th, 2010 by kafka0102
Solr的searcher autowarm(预热)有两个时机,一个是系统启动时(firstSearcher),一个是使用新的searcher替换旧的searcher时(newSearcher)。Solr支持在solrconfig.xml中对SolrCore配置SolrEventListener来实现自定义的autowarm。通常来说,Solr提供的默认实现QuerySenderListener就够用了。在我的需求中,希望solrconfig.xml中配置的SolrEventListener是针对多个SolrCore的,这要是因为我的多个SolrCore共用了一个solrconfig.xml配置。就配置autowarm的查询query来说,简单的就是配置一个常见的query,但如果系统有排序查询(sort),可以配置适宜的sort条件以预热lucene的fieldCache。下面是我自定义的SolrEventListener,效果是,如果SolrCore没有配置query,就使用default的,否则使用自己的。
实现代码修改自Solr的QuerySenderListener,代码如下:
public class TubaQuerySenderListener implements SolrEventListener { private static final Logger logger = LoggerFactory .getLogger(TubaQuerySenderListener.class); protected final SolrCore core; protected List<NamedList> queryArgs; public void init(final NamedList args) { final String coreName = core.getName(); queryArgs = (List<NamedList>)args.get(coreName); if (queryArgs == null) { queryArgs = (List<NamedList>)args.get("default"); if (queryArgs == null) { logger.warn("TubaQuerySenderListener not valid for core:"+coreName); } } logger.info("core["+coreName+"]register TubaQuerySenderListener : " + queryArgs); } public TubaQuerySenderListener(final SolrCore core) { this.core = core; } /** * Add the {@link org.apache.solr.common.params.EventParams#EVENT} with either the {@link org.apache.solr.common.params.EventParams#NEW_SEARCHER} * or {@link org.apache.solr.common.params.EventParams#FIRST_SEARCHER} values depending on the value of currentSearcher. * <p/> * Makes a copy of NamedList and then adds the parameters. * * * @param currentSearcher If null, add FIRST_SEARCHER, otherwise NEW_SEARCHER * @param nlst The named list to add the EVENT value to */ protected NamedList addEventParms(final SolrIndexSearcher currentSearcher, final NamedList nlst) { final NamedList result = new NamedList(); result.addAll(nlst); if (currentSearcher != null) { result.add(EventParams.EVENT, EventParams.NEW_SEARCHER); } else { result.add(EventParams.EVENT, EventParams.FIRST_SEARCHER); } return result; } @Override public void newSearcher(final SolrIndexSearcher newSearcher, final SolrIndexSearcher currentSearcher) { if (queryArgs == null) { return; } final SolrIndexSearcher searcher = newSearcher; for (final NamedList nlst : queryArgs) { try { // bind the request to a particular searcher (the newSearcher) final NamedList params = addEventParms(currentSearcher, nlst); final LocalSolrQueryRequest req = new LocalSolrQueryRequest(core,params) { @Override public SolrIndexSearcher getSearcher() { return searcher; } @Override public void close() { } }; final SolrQueryResponse rsp = new SolrQueryResponse(); core.execute(core.getRequestHandler(req.getParams().get(CommonParams.QT)), req, rsp); // Retrieve the Document instances (not just the ids) to warm // the OS disk cache, and any Solr document cache. Only the top // level values in the NamedList are checked for DocLists. final NamedList values = rsp.getValues(); for (int i=0; i<values.size(); i++) { final Object o = values.getVal(i); if (o instanceof DocList) { final DocList docs = (DocList)o; for (final DocIterator iter = docs.iterator(); iter.hasNext();) { newSearcher.doc(iter.nextDoc()); } } } req.close(); } catch (final Exception e) { // do nothing... we want to continue with the other requests. // the failure should have already been logged. logger.warn("",e); } } logger.info("core["+core.getName()+"]TubaQuerySenderListener newSearcher done."); } @Override public void postCommit() { throw new UnsupportedOperationException(); } }
solrconfig.xml中的配置示例如下,其中firstSearcher和newSearcher的配置是一样的,不过这不意味着它们是必须一样的。
<query> <listener event="firstSearcher" class="com.tintintech.tuba.search.TubaQuerySenderListener"> <arr name="default"> <lst> <str name="q">手机</str> <str name="start">0</str> <str name="rows">10</str> </lst> </arr> <arr name="core1"> <lst> <str name="q">手机</str> <str name="start">0</str> <str name="rows">10</str> <str name="sort">at desc</str> </lst> </arr> </listener> <listener event="newSearcher" class="com.tintintech.tuba.search.TubaQuerySenderListener"> <arr name="default"> <lst> <str name="q">手机</str> <str name="start">0</str> <str name="rows">10</str> </lst> </arr> <arr name="core1"> <lst> <str name="q">手机</str> <str name="start">0</str> <str name="rows">10</str> <str name="sort">at desc</str> </lst> </arr> </listener> </query>
=============================== 华丽的终止符 ================================
相关日志
留下评论