I was trying to implement the code,
import nltk
import nltk.tag, nltk.chunk, itertools
def ieertree2conlltags(tree, tag=nltk.tag.pos_tag):
words, ents = zip(*tree.pos())
iobs = []
prev = None
for ent in ents:
if ent == tree.node:
iobs.append('O')
prev = None
elif prev == ent:
iobs.append('I-%s' % ent)
else:
iobs.append('B-%s' % ent)
prev = ent
words, tags = zip(*tag(words))
return itertools.izip(words, tags, iobs)
def ieer_chunked_sents(tag=nltk.tag.pos_tag):
for doc in ieer.parsed_docs():
tagged = ieertree2conlltags(doc.text, tag)
yield nltk.chunk.conlltags2tree(tagged)
from chunkers import ieer_chunked_sents, ClassifierChunker
from nltk.corpus import treebank_chunk
ieer_chunks = list(ieer_chunked_sents())
chunker = ClassifierChunker(ieer_chunks[:80])
print chunker.parse(treebank_chunk.tagged_sents()[0])
score = chunker.evaluate(ieer_chunks[80:])
print score.accuracy()
It is running fine.
But as I am trying to rewrite the code as,
chunker = ClassifierChunker(list1),
where list1 is same value as,
ieer_chunks[:80]
only I am pasting the value as
[Tree('S', [Tree('LOCATION', [(u'NAIROBI', 'NNP')]), (u',', ','),
Tree('LOCATION', [(u'Kenya', 'NNP')]), (u'(', '('), Tree('ORGANIZATION',
[(u'AP', 'NNP')]), (u')', ')'), (u'_', 'NNP'), Tree('CARDINAL', [(u'Thousands',
'NNP')]), (u'of', 'IN'), (u'laborers,', 'JJ'), (u'students', 'NNS'), (u'and',
'CC'), (u'opposition', 'NN'), (u'politicians', 'NNS'), (u'on', 'IN'),
Tree('DATE', [(u'Saturday', 'NNP')]), (u'protested', 'VBD'), (u'tax', 'NN'),
(u'hikes', 'NNS'), (u'imposed', 'VBN'), (u'by', 'IN'), (u'their', 'PRP$'),
(u'cash-strapped', 'JJ'), (u'government,', 'NN'), (u'which', 'WDT'), (u'they',
'PRP'), (u'accused', 'VBD'), (u'of', 'IN'), (u'failing', 'VBG'), (u'to', 'TO'),
(u'provide', 'VB'), (u'basic', 'JJ'), (u'services.', 'NN'),....(u'(cm-kjd)',
'NN')])]
the value of whole list directly I am getting syntax error.
I tried to paste it in Python IDE outside code there also it is giving syntax
error.
If I do not paste the value and and rename ieer_chunks[:80] as list1 there is
no error.
I may be doing some problem while copying the value and pasting it.
But I did not change anything there.
Is it any error in Python part or in NLTK part?
Thanks in advance.
If any one may guide me what is the error I am doing and how may I solve it.
--
https://mail.python.org/mailman/listinfo/python-list
