Uploaded image for project: 'Terrier Core'
  1. Terrier Core
  2. TR-387

MetaIndex sometimes returns wrong/empty docno

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: 4.0
    • Fix Version/s: None
    • Component/s: .structures
    • Labels:
      None

      Description

      I use MetaIndex to get the docno:

      MetaIndex meta = index.getMetaIndex();
      String docno = meta.getItem("docno", docid);

      But it sometimes returns empty or a wrong document number which is usually close to the correct one. But in the same process, if I run it again it generally returns the correct docno.

      In order to check how often it appears, I have the following code:

      public String getDocNo(int docid) throws IOException{
      return getDocNo(docid, 0);
      }

      public String getDocNo(int docid, int callcounter) throws IOException{
      if (callcounter>50)
      return "";

      MetaIndex meta = index.getMetaIndex();
      String docno = meta.getItem("docno", docid);

      if (docno==null || docno.equals("")){
      log.warn("Fail to getdocno of docid "+ String.valueOf(docid)+" try number "+String.valueOf(callcounter));
      docno=getDocNo(docid, callcounter+1);
      }
      else {
      //checking correctness of docno!!
      String docno_again = meta.getItem("docno", docid);
      if (!docno_again.equals(docno)){
      log.warn("Inconsistency in the fetched DocNos ("+String.valueOf(docno)+" vs "+String.valueOf(docno_again)+
      ") of docid "+ String.valueOf(docid)+" try number "+String.valueOf(callcounter));
      docno=getDocNo(docid, callcounter+1);
      }
      }
      return docno;
      }

      As a note, I am running the getDocNo procedure from different threads while the index object is static.

      Regards
      Navid

        Attachments

          Activity

          Hide
          craigm Craig Macdonald added a comment -

          Hi,

          You cant rely on any thread-safety for Terrier's index structures. I would think if you try synchronising on the metaindex, the problem will go away.

          Craig

          Show
          craigm Craig Macdonald added a comment - Hi, You cant rely on any thread-safety for Terrier's index structures. I would think if you try synchronising on the metaindex, the problem will go away. Craig
          Hide
          craigm Craig Macdonald added a comment -

          thread-safety of index structures is not supported.

          Show
          craigm Craig Macdonald added a comment - thread-safety of index structures is not supported.

            People

            • Assignee:
              craigm Craig Macdonald
              Reporter:
              rekabsaz Navid Rekabsaz
            • Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: