Hi, Found some typos. In package description: http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/analysis/package-summary.html?is-external=true#package_description
In the example code bellow the following corrections are needed: OffsetAttribute offsetAtt = addAttribute(OffsetAttribute.class); *** Should be: OffsetAttribute offsetAtt = ts.addAttribute(OffsetAttribute.class); *** And: } finally { ts.close(); // Release resources associated with this stream. } *** Should be: } finally { analyzer.close(); ts.close(); // Release resources associated with this stream. } Example code: Version matchVersion = Version.LUCENE_XY; // Substitute desired Lucene version for XY Analyzer analyzer = new StandardAnalyzer(matchVersion); // or any other analyzer TokenStream ts = analyzer.tokenStream("myfield", new StringReader("some text goes here")); OffsetAttribute offsetAtt = addAttribute(OffsetAttribute.class); try { ts.reset(); // Resets this stream to the beginning. (Required) while (ts.incrementToken()) { // Use AttributeSource.reflectAsString(boolean) <http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/util/AttributeSource.html#reflectAsString(boolean)> // for token stream debugging. System.out.println("token: " + ts.reflectAsString(true)); System.out.println("token start offset: " + offsetAtt.startOffset()); System.out.println(" token end offset: " + offsetAtt.endOffset()); } ts.end(); // Perform end-of-stream operations, e.g. set the final offset. } finally { ts.close(); // Release resources associated with this stream. }