Re: [Twisted-Python] Embedding manhole interpreter into insults widget.
On 6 August 2014 22:48, Glyph Lefkowitz wrote: > > Optimizing this will almost certainly involve patching Twisted. Which you > should definitely do :). > > The reason the entire screen is redrawn is that a *general* implementation > of a widget, i.e. one that can be displayed anywhere on the terminal, you > do just have to redraw the entire widget when certain things happen, like > when the screen scrolls. You can see this in other software by observing > the fairly significant performance difference between a vertical split and > a horizontal split in tmux. Nothing to do with Twisted there, just a > general limitation of terminals. > > The optimizations you could implement are one where inputting individual > characters doesn't cause a full screen redraw, and avoiding a full screen > redraw when scrolling if the edges of the widget touch the edges of the > full terminal (setting up a scroll area for that case). > When manhole is used directly, then it is very responsive, and the performance doesn't degrade with increase of screen size. It holds only what user entered and writes the input line and command output directly to its transport, without bothering itself with processing of the whole screen. When I embed manhole into the widget like above, then the performance suffers. I think this is due to `TerminalBuffer` having to hold whole widget area (columns * rows) in memory, even if only a small portion of it is actually occupied by user input and command output. Rendering the whole buffer on each keystroke is slow. After experimenting a little bit, I think optimizations that you described mostly apply to `ManholeWidget.render` method, which shouldn't traverse the whole buffer on each keystroke. Currently I don't see how this involves patching widget implementation in Twisted. What is more important to me is that `TerminalBuffer` implementation is slightly incomplete. I think it misses color support, because I couldn't make it work with `ColoredManhole`. > > -glyph > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > -- Regards, Maxim ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Embedding manhole interpreter into insults widget.
On 6 Aug, 06:14 pm, lacrima.ma...@gmail.com wrote: Hello, I have been trying to create a widget that encloses manhole interpreter. Here is somewhat hacky implementation that I came up with at this moment: Widgets and the manhole interpreter are implemented using two pretty distinct approaches. As you've figured out, these approaches can be integrated (at least in one direction) using a terminal emulator - `twisted.conch.insults.helper.TerminalBuffer`. However, `TerminalBuffer` is an extremely low quality terminal emulator. Among its many flaws is that it uses a number of incredibly inefficient implementations for common terminal operations (many of its other flaws are along the lines of "it has a bug in the implementation of X"). If I were going to create a manhole widget then I would do it "natively" - not by bridging the existing manhole protocol to the widget system with TerminalBuffer but by writing a new widget that does manhole stuff all by itself. I suggest that you might want to pursue this approach instead of trying to optimize and fix bugs in TerminalBuffer. Ultimately it would be nice if a widget-based manhole were *the* manhole and the current implementation went away. However, that said, as far as I know, no vt-style terminals support efficient redrawing for the "vertically scroll one portion of the terminal separated from another portion by a vertical divider". You will be able to avoid the redundant work of re-rendering the Python buffer representing the terminal state but you'll still have to re- transmit the bytes representing the entire terminal display to the terminal on each redraw - unless you avoid vertical dividers. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Is there pb documentation somewhere?
glyph, > I really wish we would stop calling things "bad" and "good". My wording of exarkun's wording. He gave a much more detailed description of what he think's is "crazy" about pb. > make your own decisions about how to write your own code. Indeed, but gathering information from wiser folks is always a good idea, and usually best done _often_ during development :) > I'm happy to trade 2-for-1 - if you do two code reviews, I will regard it as an immediate obligation for > me to review a ticket you direct me to ;). Deal. However, rather than direct your attention to tickets, at this stage I would rather trade reviews for discussion. I'll do two reviews and then post a few questions to this mailing list thread. Once I start actually writing patches/new code we can trade reviews for attention to tickets. Ok? > These would also be easier to land, and a couple of decades in open source has taught me that nothing > motivates development activity like successful development activity ;). Indeed. There are one or two architectural issues I want to understand before moving on to real coding. I will try to get through that asap by reviewing tickets and trading for discussion of those architectural issues. > Hopefully you can make sense out of the explanations above and your own existing knowledge. > Are there any other phases of the process which are confusing? This all makes sense now. I hadn't understood the point of the cooker, but now that you've explained it, I understand what's going on. I will transform your mailing list explanation to documentation shortly. Daniel ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] pb.Copyable knowledge barrier
goal: Give knowitall.py awareness of wordsmith.py addition.
to run
./library.sh # opens 7999 for remote_dictionary and remote_define methods
then both:
python knowitall.py
python wordsmith.py
library.sh due to main/jelly issues
from twisted.spread import pb
class UknownWordException(Exception):
pass
class UnknownDefinition(UknownWordException):
pass
# would like to remote define via error
class Dictionary():
def __init__(self, language, word=None, definition=None):
self.language = language
self.words = {}
if not word:
word = language
if not definition:
definition = language + ' is a word worthy of a language'
self.words[word] = set([definition])
print self.words
def get_language(self):
return self.language
def lookup_word(self, word, definition = None):
print 'lookup_word', self.words
try:
return self.words[word]
except:
if definition:
raise UknownWordException(word)
else:
raise UnknownDefinition(word, definition)
class CopyDictionary(Dictionary, pb.Copyable):
def getStateToCopy(self):
print 'state getting copied', self.language
return self.language, self.words
class DictionaryEditor(pb.Referenceable):
def remote_word_definition(self, lanague, word, definition):
pass
class RemoteDictionary(pb.RemoteCopy, Dictionary):
def setCopyableState(self, state):
print 'remote state receives copy', state
self.language = state[0]
self.words = state[1]
class Library(pb.Root):
dictionaries = []
def remote_define(self, language, word, definition):
dictionary = self.remote_dictionary(language, word, definition)
try:
definition_set = dictionary.lookup_word(word, definition)
print 'got definition_set', definition_set
if definition not in definition_set:
print 'new definition'
definition_set.add(definition)
else:
print 'existing definition'
except Exception as e:
print e
print 'new word'
dictionary.words[word] = set([definition])
return dictionary.words
def remote_dictionary(self, language, word = None, definition = None):
for d in Library.dictionaries:
if d.get_language() == language:
print 'remote'
return d
print 'create copyable version'
dictionary = CopyDictionary(language, word, definition)
Library.dictionaries.append(dictionary)
return dictionary
from twisted.spread import pb
from example import CopyDictionary, RemoteDictionary
from twisted.internet import reactor
from twisted.internet import defer
factory = pb.PBClientFactory()
reactor.connectTCP("localhost", 7999, factory)
pb.setUnjellyableForClass(CopyDictionary, RemoteDictionary)
def use_dictionary(dictionary):
for w in dictionary.words.keys():
for d in dictionary.words[w]:
print w, d
reactor.callLater(10, use_dictionary, dictionary)
def goto_library():
return factory.getRootObject()
import sys
language = 'english'
if len(sys.argv) > 1:
language = sys.argv[1]
def test():
d = goto_library()
d.addCallback(lambda root: root.callRemote('dictionary', language))
d.addCallback(use_dictionary)
return d
reactor.callWhenRunning(test)
reactor.run()
library.sh
Description: Bourne shell script
from twisted.spread import pb
from twisted.internet import reactor
from twisted.internet import defer
factory = pb.PBClientFactory()
reactor.connectTCP("localhost", 7999, factory)
def goto_library():
return factory.getRootObject()
def define_result(ans, language, word, definition):
# audit submission later
print ans
reactor.stop()
language = 'english'
word = language
definition = word + ' is part of ' + language
import sys
if len(sys.argv) > 1:
language = sys.argv[1]
if len(sys.argv) > 2:
word = sys.argv[2]
if len(sys.argv) > 3:
definition = sys.argv[3]
def test():
d = goto_library()
d.addCallback(lambda root: root.callRemote('define', language, word, definition))
d.addCallback(define_result, language, word, definition)
return d
reactor.callWhenRunning(test)
reactor.run()
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] pb.Copyable knowledge barrier
wordsmith argument order issues re-take.
Cheers,
Kevin
On Thu, Aug 7, 2014 at 11:29 PM, Kevin Mcintyre wrote:
> goal: Give knowitall.py awareness of wordsmith.py addition.
>
> to run
> ./library.sh # opens 7999 for remote_dictionary and remote_define methods
> then both:
> python knowitall.py
> python wordsmith.py
>
> library.sh due to main/jelly issues
>
>
from twisted.spread import pb
class UknownWordException(Exception):
pass
class UnknownDefinition(UknownWordException):
pass
# would like to remote define via error
class Dictionary():
def __init__(self, language, word=None, definition=None):
self.language = language
self.words = {}
if not word:
word = language
if not definition:
definition = language + ' is a word worthy of a language'
self.words[word] = set([definition])
print self.words
def get_language(self):
return self.language
def lookup_word(self, word, definition = None):
print 'lookup_word', self.words
try:
return self.words[word]
except:
if definition:
raise UknownWordException(word)
else:
raise UnknownDefinition(word, definition)
class CopyDictionary(Dictionary, pb.Copyable):
def getStateToCopy(self):
print 'state getting copied', self.language
return self.language, self.words
class DictionaryEditor(pb.Referenceable):
def remote_word_definition(self, lanague, word, definition):
pass
class RemoteDictionary(pb.RemoteCopy, Dictionary):
def setCopyableState(self, state):
print 'remote state receives copy', state
self.language = state[0]
self.words = state[1]
class Library(pb.Root):
dictionaries = []
def remote_define(self, language, word, definition):
dictionary = self.remote_dictionary(language, word, definition)
try:
definition_set = dictionary.lookup_word(word, definition)
print 'got definition_set', definition_set
if definition not in definition_set:
print 'new definition'
definition_set.add(definition)
else:
print 'existing definition'
except Exception as e:
print e
print 'new word'
dictionary.words[word] = set([definition])
return dictionary.words
def remote_dictionary(self, language, word = None, definition = None):
for d in Library.dictionaries:
if d.get_language() == language:
print 'remote'
return d
print 'create copyable version'
dictionary = CopyDictionary(language, word, definition)
Library.dictionaries.append(dictionary)
return dictionary
from twisted.spread import pb
from example import CopyDictionary, RemoteDictionary
from twisted.internet import reactor
from twisted.internet import defer
factory = pb.PBClientFactory()
reactor.connectTCP("localhost", 7999, factory)
pb.setUnjellyableForClass(CopyDictionary, RemoteDictionary)
def use_dictionary(dictionary):
for w in dictionary.words.keys():
for d in dictionary.words[w]:
print w, d
reactor.callLater(10, use_dictionary, dictionary)
def goto_library():
return factory.getRootObject()
import sys
language = 'english'
if len(sys.argv) > 1:
language = sys.argv[1]
def test():
d = goto_library()
d.addCallback(lambda root: root.callRemote('dictionary', language))
d.addCallback(use_dictionary)
return d
reactor.callWhenRunning(test)
reactor.run()
library.sh
Description: Bourne shell script
from twisted.spread import pb
from twisted.internet import reactor
from twisted.internet import defer
factory = pb.PBClientFactory()
reactor.connectTCP("localhost", 7999, factory)
def goto_library():
return factory.getRootObject()
def define_result(ans, language, word, definition):
# audit submission later
print ans
reactor.stop()
import sys
language = 'english'
if len(sys.argv) > 1:
language = sys.argv[1]
word = language
if len(sys.argv) > 2:
word = sys.argv[2]
definition = word + ' is part of ' + language
if len(sys.argv) > 3:
definition = sys.argv[3]
print 'language', language
print 'word', word
print 'definition', definition
def test():
d = goto_library()
d.addCallback(lambda root: root.callRemote('define', language, word, definition))
d.addCallback(define_result, language, word, definition)
return d
reactor.callWhenRunning(test)
reactor.run()
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Embedding manhole interpreter into insults widget.
Hi, Thanks for your explanation. Now I understand a lot better how things work. I will try to implement "native" manhole widget. I think this will be a good exercise for me to learn how terminals and `twisted.conch` work. Thanks. On 7 August 2014 14:53, wrote: > On 6 Aug, 06:14 pm, lacrima.ma...@gmail.com wrote: > >> Hello, >> >> I have been trying to create a widget that encloses manhole interpreter. >> Here is somewhat hacky implementation that I came up with at this moment: >> > > Widgets and the manhole interpreter are implemented using two pretty > distinct approaches. > > As you've figured out, these approaches can be integrated (at least in one > direction) using a terminal emulator - `twisted.conch.insults.helper. > TerminalBuffer`. > > However, `TerminalBuffer` is an extremely low quality terminal emulator. > Among its many flaws is that it uses a number of incredibly inefficient > implementations for common terminal operations (many of its other flaws are > along the lines of "it has a bug in the implementation of X"). > > If I were going to create a manhole widget then I would do it "natively" - > not by bridging the existing manhole protocol to the widget system with > TerminalBuffer but by writing a new widget that does manhole stuff all by > itself. > > I suggest that you might want to pursue this approach instead of trying to > optimize and fix bugs in TerminalBuffer. > > Ultimately it would be nice if a widget-based manhole were *the* manhole > and the current implementation went away. > > However, that said, as far as I know, no vt-style terminals support > efficient redrawing for the "vertically scroll one portion of the terminal > separated from another portion by a vertical divider". > > You will be able to avoid the redundant work of re-rendering the Python > buffer representing the terminal state but you'll still have to re- > transmit the bytes representing the entire terminal display to the terminal > on each redraw - unless you avoid vertical dividers. > > Jean-Paul > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > -- Regards, Maxim ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
