IIRC first versions of patches that added payloads support had this notion of payload by field rather than by token, but later it was modified to be by token only.
I have seen two code patterns to add payloads to tokens. The first one created the field text with a reserved separator/delimiter which was later identified by the analyzer who separated the payload part from the token part, created the token and set the payload. The other pattern was to create a field with a TokenStream. Can be done only for non storable fields. Here you can create the token in advance, and you have a SingleTokenStream (I think this is how it is called) to wrap it in case it is a single token. Since the token is created in advance, there's no analysis going on, and you can set the payload of that token on the spot.I prefer this pattern - more efficient and elegant. Doron On Thu, Aug 14, 2008 at 6:14 AM, Antony Bowesman <[EMAIL PROTECTED]> wrote: > I started playing with payloads and have been trying to work out how to get > the data into the payload > > I have a field where I want to add the following untokenized fields > > A1 > A2 > A3 > > With these fields, I would like to add the payloads > > B1 > B2 > B3 > > Firstly, it looks like you cannot add payloads to untokenized fields. Is > this correct? In my usage, A and B are simply external Ids so must not be > tokenized and there is always a 1-->1 relationship between them. > > Secondly, what is the way to provide the payload data to the tokenizer. It > looks like I have to add a List/Map of payload data to a custom Tokenizer > and Analyzer, which is then consumed each "next(Token)". However, it would > be nice if, in my use case, I could use some kind of construct like: > > Document doc = new Document() > Field f = new Field("myField", "A1", Field.Store.NO, > Field.Index.UNTOKENIZED); > f.setPayload("B1"); > doc.add(f); > > and avoid the whole unnecessary Tokenizer/Analyzer overhead and give > support for payloads in untokenized fields. > > It looks like it would be trivial to implement in > DocumentsWriter.invertField(). Or would this corrupt the Fieldable > interface in an undesirable way? > > Antony > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >