Well, ideally I would first try to cause the pty to block if possible,
but I have tried to do this on several occasions. It is difficult to get
the limit right because tmux can't estimate the actual network latency
or how much is buffered on the network, so the high water mark has to be
conservative. Still, maybe my approach was flawed, so by all means have
a go and see if you can get it to work - it might work better when you
control the receiving client and are taking input data directly.
I've attached a diff I was playing with a while back but it probably
won't apply anymore. This time-based reading rather than based on the
size of the output buffer, but the changes to use an event rather than
bufferevent might be relevant.
On Tue, Apr 10, 2012 at 10:06:12AM -0700, George Nachman wrote:
> There is a problem with buffering in control mode. This happens when a pty
> produces output faster than a control client can read. The problems are:
> - the tmux job grows in memory usage with no upper bound
> - even an alert user can't stop an out-of-control job fast enough to avoid
> having to wait for many megabytes of output (tmux can read from the pty
> very fast, but ssh can be very slow)
> To illustrate, I ran "yes" for 3 seconds over ssh and then pressed ^C. It
> took 45 seconds to receive the buffered output. *
> Possible solutions are:
> 1. Limit the amount of buffering that is done for control clients and
> force clients to reload state
> 2. Cause programs that produce too much output to block
> 3. Provide clients with a way to force the buffer to be flushed
> Taking each in turn:
> 1. Limit the amount of buffering that is done for control clients and
> force clients to reload state
> When a control client has too much data in its write buffer, we would stop
> sending it new spontaneous messages. For instance, tmux could issue a
> %overflow message to indicate this to the client and then refuse to queue
> for writing all future %output and other messages. When the client
> receives %overflow, it must reinitialize its state (get window layouts and
> update them if they've changed, re-download missing history and issue a
> "control set-ready" to start up again.
> Pros:*
> - Works automatically with no user interaction
> - Doesn't affect non-control clients
> Cons:
> - Would introduce a very large delay when it triggers--ALL window panes'
> partial histories must be refetched
> - In pathological cases, downloading history might be worse than receiving
> the output
> - Buffer too small means this triggers all the time. Buffer too large
> means it doesn't trigger enough and the problem of stopping an
> out-of-control job remains. If you have lots of moderately talkative
> window panes, you can't make the buffer small enough.
> - Users would need to constantly tweak & tune the buffer size, but many
> wouldn't know about it at all.
> 2. Cause PTYs that produce too much output to block
> This is the traditional way of throttling overly chatty programs, and it
> works well because it pushes the pain back to the misbehaving program. A
> coarse implementation would stop reading from all ptys when any control
> client has too much data in its output buffer, and would resume reading
> when they all fall under some low water mark. A better implementation
> would keep track of how much output is buffered from each pty and block
> only those that are above a high water mark (not sure if this would
> be*noticeably*better though)
> Pros:
> - Works automatically with no user interaction
> - Good user experience for control client users
> Cons:
> - All clients become as slow as the slowest control client
> 3. Provide clients with a way to force the buffer to be flushed
> When a user sees that there is a program is producing output too quickly,
> he could issue a command that flushes the buffers. This is like manually
> invoking option #1.*
> Pros:*
> - Predictable behavior
> - Doesn't affect non-control clients
> Cons:
> - Would introduce a very large delay when it is triggered--ALL window
> panes' partial histories must be refetched
> - The tmux process's memory usage can grow without bound if nobody is at
> the keyboard
> - Requires manual intervention, not obvious to most users
> I think #2 is the most sensible solution. It's fairly simple to implement,
> works automatically, will usually go unnoticed by the user, and in the
> worst case is*mitigable*by the user detaching the slow client. Thoughts?
> ------------------------------------------------------------------------------
> Better than sec? Nothing is better than sec when it comes to
> monitoring Big Data applications. Try Boundary one-second
> resolution app monitoring today. Free.
> http://p.sf.net/sfu/Boundary-dev2dev
> _______________________________________________
> tmux-users mailing list
> tmux-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/tmux-users
Index: cmd-paste-buffer.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/cmd-paste-buffer.c,v
retrieving revision 1.17
diff -u -p -r1.17 cmd-paste-buffer.c
--- cmd-paste-buffer.c 3 Mar 2012 09:43:22 -0000 1.17
+++ cmd-paste-buffer.c 9 Mar 2012 21:17:28 -0000
@@ -113,19 +113,19 @@ cmd_paste_buffer_filter(struct window_pa
size_t seplen;
if (bracket)
- bufferevent_write(wp->event, "\033[200~", 6);
+ window_pane_write(wp, "\033[200~", 6);
seplen = strlen(sep);
while ((lf = memchr(data, '\n', end - data)) != NULL) {
if (lf != data)
- bufferevent_write(wp->event, data, lf - data);
- bufferevent_write(wp->event, sep, seplen);
+ window_pane_write(wp, data, lf - data);
+ window_pane_write(wp, sep, seplen);
data = lf + 1;
}
if (end != data)
- bufferevent_write(wp->event, data, end - data);
+ window_pane_write(wp, data, end - data);
if (bracket)
- bufferevent_write(wp->event, "\033[201~", 6);
+ window_pane_write(wp, "\033[201~", 6);
}
Index: cmd-pipe-pane.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/cmd-pipe-pane.c,v
retrieving revision 1.21
diff -u -p -r1.21 cmd-pipe-pane.c
--- cmd-pipe-pane.c 27 Oct 2011 22:41:03 -0000 1.21
+++ cmd-pipe-pane.c 9 Mar 2012 21:17:28 -0000
@@ -120,7 +120,7 @@ cmd_pipe_pane_exec(struct cmd *self, str
close(pipe_fd[1]);
wp->pipe_fd = pipe_fd[0];
- wp->pipe_off = EVBUFFER_LENGTH(wp->event->input);
+ wp->pipe_off = EVBUFFER_LENGTH(wp->read_buf);
wp->pipe_event = bufferevent_new(wp->pipe_fd,
NULL, NULL, cmd_pipe_pane_error_callback, wp);
Index: input-keys.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/input-keys.c,v
retrieving revision 1.23
diff -u -p -r1.23 input-keys.c
--- input-keys.c 30 Jul 2011 18:01:26 -0000 1.23
+++ input-keys.c 9 Mar 2012 21:17:29 -0000
@@ -151,9 +151,9 @@ input_key(struct window_pane *wp, int ke
*/
if (key != KEYC_NONE && (key & ~KEYC_ESCAPE) < 0x100) {
if (key & KEYC_ESCAPE)
- bufferevent_write(wp->event, "\033", 1);
+ window_pane_write(wp, "\033", 1);
ch = key & ~KEYC_ESCAPE;
- bufferevent_write(wp->event, &ch, 1);
+ window_pane_write(wp, &ch, 1);
return;
}
@@ -163,7 +163,7 @@ input_key(struct window_pane *wp, int ke
*/
if (options_get_number(&wp->window->options, "xterm-keys")) {
if ((out = xterm_keys_lookup(key)) != NULL) {
- bufferevent_write(wp->event, out, strlen(out));
+ window_pane_write(wp, out, strlen(out));
xfree(out);
return;
}
@@ -194,8 +194,8 @@ input_key(struct window_pane *wp, int ke
/* Prefix a \033 for escape. */
if (key & KEYC_ESCAPE)
- bufferevent_write(wp->event, "\033", 1);
- bufferevent_write(wp->event, ike->data, dlen);
+ window_pane_write(wp, "\033", 1);
+ window_pane_write(wp, ike->data, dlen);
}
/* Translate mouse and output. */
@@ -220,7 +220,7 @@ input_mouse(struct window_pane *wp, stru
buf[len++] = m->x + 33;
buf[len++] = m->y + 33;
}
- bufferevent_write(wp->event, buf, len);
+ window_pane_write(wp, buf, len);
} else if ((m->b & MOUSE_BUTTON) != MOUSE_2) {
value = options_get_number(&wp->window->options, "mode-mouse");
if (value == 1 &&
Index: input.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/input.c,v
retrieving revision 1.49
diff -u -p -r1.49 input.c
--- input.c 3 Mar 2012 09:43:22 -0000 1.49
+++ input.c 9 Mar 2012 21:17:30 -0000
@@ -704,7 +704,7 @@ input_parse(struct window_pane *wp)
{
struct input_ctx *ictx = &wp->ictx;
const struct input_transition *itr;
- struct evbuffer *evb = wp->event->input;
+ struct evbuffer *evb = wp->read_buf;
u_char *buf;
size_t len, off;
@@ -827,7 +827,7 @@ input_reply(struct input_ctx *ictx, cons
vasprintf(&reply, fmt, ap);
va_end(ap);
- bufferevent_write(ictx->wp->event, reply, strlen(reply));
+ window_pane_write(ictx->wp, reply, strlen(reply));
xfree(reply);
}
Index: options-table.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/options-table.c,v
retrieving revision 1.23
diff -u -p -r1.23 options-table.c
--- options-table.c 29 Feb 2012 21:10:51 -0000 1.23
+++ options-table.c 9 Mar 2012 21:17:30 -0000
@@ -569,6 +569,13 @@ const struct options_table_entry window_
.default_num = 0
},
+ { .name = "rate-limit",
+ .type = OPTIONS_TABLE_NUMBER,
+ .minimum = 0,
+ .maximum = UINT_MAX,
+ .default_num = 0
+ },
+
{ .name = "remain-on-exit",
.type = OPTIONS_TABLE_FLAG,
.default_num = 0
Index: server-fn.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/server-fn.c,v
retrieving revision 1.53
diff -u -p -r1.53 server-fn.c
--- server-fn.c 29 Jan 2012 02:22:11 -0000 1.53
+++ server-fn.c 9 Mar 2012 21:17:31 -0000
@@ -337,7 +337,9 @@ server_destroy_pane(struct window_pane *
old_fd = wp->fd;
if (wp->fd != -1) {
- bufferevent_free(wp->event);
+ event_del(&wp->read_ev);
+ if (wp->flags & PANE_WRITING)
+ event_del(&wp->write_ev);
close(wp->fd);
wp->fd = -1;
}
Index: tmux.1
===================================================================
RCS file: /cvs/src/usr.bin/tmux/tmux.1,v
retrieving revision 1.279
diff -u -p -r1.279 tmux.1
--- tmux.1 4 Mar 2012 20:50:53 -0000 1.279
+++ tmux.1 9 Mar 2012 21:17:32 -0000
@@ -2556,6 +2556,12 @@ Like
.Ic base-index ,
but set the starting index for pane numbers.
.Pp
+.It Ic rate-limit Ar bytes
+Limit output rate from any single pane to
+.Ar bytes .
+This option can be used to attempt to prevent excessive buffering of
+fast output when the producer can be faster than the consuming terminal.
+.Pp
.It Xo Ic remain-on-exit
.Op Ic on | off
.Xc
Index: tmux.h
===================================================================
RCS file: /cvs/src/usr.bin/tmux/tmux.h,v
retrieving revision 1.319
diff -u -p -r1.319 tmux.h
--- tmux.h 9 Mar 2012 09:57:40 -0000 1.319
+++ tmux.h 9 Mar 2012 21:17:33 -0000
@@ -814,6 +814,7 @@ struct window_pane {
int flags;
#define PANE_REDRAW 0x1
+#define PANE_WRITING 0x2
char *cmd;
char *shell;
@@ -823,7 +824,11 @@ struct window_pane {
char tty[TTY_NAME_MAX];
int fd;
- struct bufferevent *event;
+ struct event read_ev;
+ struct evbuffer* read_buf;
+ struct timeval read_last;
+ struct event write_ev;
+ struct evbuffer* write_buf;
struct input_ctx ictx;
@@ -1943,6 +1948,8 @@ void window_set_active_pane(struct win
struct window_pane *window_add_pane(struct window *, u_int);
void window_resize(struct window *, u_int, u_int);
void window_remove_pane(struct window *, struct window_pane *);
+void window_pane_queue_write(struct window_pane *);
+void window_pane_write(struct window_pane *, const void *, size_t);
struct window_pane *window_pane_at_index(struct window *, u_int);
struct window_pane *window_pane_next_by_number(struct window *,
struct window_pane *, u_int);
Index: window-copy.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/window-copy.c,v
retrieving revision 1.76
diff -u -p -r1.76 window-copy.c
--- window-copy.c 4 Dec 2011 16:18:01 -0000 1.76
+++ window-copy.c 9 Mar 2012 21:17:34 -0000
@@ -174,8 +174,11 @@ window_copy_init(struct window_pane *wp)
data->searchtype = WINDOW_COPY_OFF;
data->searchstr = NULL;
- if (wp->fd != -1)
- bufferevent_disable(wp->event, EV_READ|EV_WRITE);
+ if (wp->fd != -1) {
+ event_del(&wp->read_ev);
+ if (wp->flags & PANE_WRITING)
+ event_del(&wp->write_ev);
+ }
data->jumptype = WINDOW_COPY_OFF;
data->jumpchar = '\0';
@@ -237,8 +240,11 @@ window_copy_free(struct window_pane *wp)
{
struct window_copy_mode_data *data = wp->modedata;
- if (wp->fd != -1)
- bufferevent_enable(wp->event, EV_READ|EV_WRITE);
+ if (wp->fd != -1) {
+ event_add(&wp->read_ev, NULL);
+ if (wp->flags & PANE_WRITING)
+ event_add(&wp->write_ev, NULL);
+ }
if (data->searchstr != NULL)
xfree(data->searchstr);
Index: window.c
===================================================================
RCS file: /cvs/src/usr.bin/tmux/window.c,v
retrieving revision 1.72
diff -u -p -r1.72 window.c
--- window.c 2 Feb 2012 00:10:12 -0000 1.72
+++ window.c 9 Mar 2012 21:17:34 -0000
@@ -61,8 +61,8 @@ struct window_pane_tree all_window_panes
u_int next_window_pane_id;
u_int next_window_id;
-void window_pane_read_callback(struct bufferevent *, void *);
-void window_pane_error_callback(struct bufferevent *, short, void *);
+void window_pane_read_callback(int, short, void *);
+void window_pane_write_callback(int, short, void *);
RB_GENERATE(winlinks, winlink, entry, winlink_cmp);
@@ -615,7 +615,6 @@ window_pane_create(struct window *w, u_i
wp->cwd = NULL;
wp->fd = -1;
- wp->event = NULL;
wp->mode = NULL;
@@ -647,7 +646,9 @@ window_pane_destroy(struct window_pane *
window_pane_reset_mode(wp);
if (wp->fd != -1) {
- bufferevent_free(wp->event);
+ event_del(&wp->read_ev);
+ if (wp->flags & PANE_WRITING)
+ event_del(&wp->write_ev);
close(wp->fd);
}
@@ -683,7 +684,9 @@ window_pane_spawn(struct window_pane *wp
struct termios tio2;
if (wp->fd != -1) {
- bufferevent_free(wp->event);
+ event_del(&wp->read_ev);
+ if (wp->flags & PANE_WRITING)
+ event_del(&wp->write_ev);
close(wp->fd);
}
if (cmd != NULL) {
@@ -755,30 +758,82 @@ window_pane_spawn(struct window_pane *wp
setblocking(wp->fd, 0);
- wp->event = bufferevent_new(wp->fd,
- window_pane_read_callback, NULL, window_pane_error_callback, wp);
- bufferevent_enable(wp->event, EV_READ|EV_WRITE);
+ event_set(&wp->read_ev,
+ wp->fd, EV_READ|EV_PERSIST, window_pane_read_callback, wp);
+ if ((wp->read_buf = evbuffer_new ()) == NULL)
+ fatalx("evbuffer_new failed");
+
+ event_set(&wp->write_ev,
+ wp->fd, EV_WRITE|EV_PERSIST, window_pane_write_callback, wp);
+ if ((wp->write_buf = evbuffer_new ()) == NULL)
+ fatalx("evbuffer_new failed");
+
+ if (gettimeofday (&wp->read_last, NULL) != 0)
+ fatal("gettimeofday failed");
+ event_add(&wp->read_ev, NULL);
return (0);
}
+void
+window_pane_write(struct window_pane *wp, const void *buf, size_t len)
+{
+ evbuffer_add(wp->write_buf, buf, len);
+ window_pane_queue_write(wp);
+}
+
+void
+window_pane_queue_write(struct window_pane *wp)
+{
+ if (wp->flags & PANE_WRITING)
+ return;
+ event_add(&wp->write_ev, NULL);
+ wp->flags |= PANE_WRITING;
+}
+
/* ARGSUSED */
void
-window_pane_read_callback(unused struct bufferevent *bufev, void *data)
+window_pane_read_callback(unused int fd, unused short events, void* data)
{
struct window_pane *wp = data;
+ u_int limit;
+ struct timeval tv;
+ struct timeval tvdiff;
+ double last;
+ size_t size;
char *new_data;
size_t new_size;
- new_size = EVBUFFER_LENGTH(wp->event->input) - wp->pipe_off;
+ if (gettimeofday(&tv, NULL) != 0)
+ fatal("gettimeofday failed");
+
+ limit = options_get_number (&wp->window->options, "rate-limit");
+ if (limit != 0) {
+ timersub(&tv, &wp->read_last, &tvdiff);
+ last = tvdiff.tv_sec + (tvdiff.tv_usec / 1e6);
+
+ size = limit * last;
+ if (size > limit)
+ size = limit;
+ if (size == 0)
+ return;
+ } else
+ size = SIZE_MAX;
+ memcpy(&wp->read_last, &tv, sizeof wp->read_last);
+ if (evbuffer_read(wp->read_buf, wp->fd, size) <= 0) {
+ server_destroy_pane(wp);
+ return;
+ }
+
+ new_size = EVBUFFER_LENGTH(wp->read_buf) - wp->pipe_off;
if (wp->pipe_fd != -1 && new_size > 0) {
- new_data = EVBUFFER_DATA(wp->event->input);
+ new_data = EVBUFFER_DATA(wp->read_buf);
bufferevent_write(wp->pipe_event, new_data, new_size);
}
input_parse(wp);
- wp->pipe_off = EVBUFFER_LENGTH(wp->event->input);
+ wp->pipe_off = EVBUFFER_LENGTH(wp->read_buf);
/*
* If we get here, we're not outputting anymore, so set the silence
@@ -791,12 +846,19 @@ window_pane_read_callback(unused struct
/* ARGSUSED */
void
-window_pane_error_callback(
- unused struct bufferevent *bufev, unused short what, void *data)
+window_pane_write_callback(unused int fd, unused short events, void* data)
{
struct window_pane *wp = data;
- server_destroy_pane(wp);
+ if (evbuffer_write(wp->write_buf, wp->fd) <= 0) {
+ server_destroy_pane(wp);
+ return;
+ }
+
+ if (EVBUFFER_LENGTH(wp->write_buf) == 0) {
+ event_del(&wp->write_ev);
+ wp->flags &= ~PANE_WRITING;
+ }
}
void
------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev
_______________________________________________
tmux-users mailing list
tmux-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/tmux-users
