[9fans] Migration of vac score to fossil

[Ole-Hjalmar Kristensen]

I have a file server running Linux at home, with a normal ext3 file system
and a plan9port venti. I use this venti for vac backup of both the ext3
file system and other Linux boxes. However, the 2 TB ext3 is running out of
space, while the venti is roughly 50% full. I could just buy a bigger disk,
of course, but the exit3 itself is mainly an archive (pictures, video, and
many years of accumulated documents and software), so I consider switching
to a fossil+venti file server instead. The fossil manual says "The score
should have been generated by fossil rather than by vac, so that the
appropriate snapshot metadata is present". Is there any way of coercing
fossil to initialize itself properly from a score produced by vac? I could
copy the files from ext3, but would likely run out of space in fossil,
which I gather is a bad idea. I run a 9front cwfs auth/cpu/file server, but
have no experience with fossil, so any help is welcome.

Re: [9fans] Migration of vac score to fossil

[Ole-Hjalmar Kristensen]

Yes, I was think about something like that. As an initial experiment, I
tried to used flfmt with the -v option on a vac archive just to see the
result. It seems that the vac archive does not contain the max qid that
flfmt needs. This seems strange to me, as vac -a should need this info just
as much as fossil needs it. Maybe it's tucked away somewhere else. Guess I
need to look some more at the code.



ole@ole-TECRA-R940 ~/Desktop/plan9 $ bin/fossil/flfmt -h 192.168.0.101 -v
f648dbae0075eb73bc394ad6cd4c059e655e127c fossil.dat
fs header block already exists; are you sure? [y/n]: y
fs file is mounted via devmnt (is not a kernel device); are you sure?
[y/n]: y
0xfb1e734c
0x1d1feaf1
c85978546e4048fce83120d3992cfc2f57ff2f8c
bin/fossil/flfmt: bad root: no qidSpace

/*
 * Maximum qid is recorded in root's msource, entry #2 (conveniently in
e).
 */
ventiRead(e.score, VtDataType);
if(!mbUnpack(&mb, buf, bsize))
sysfatal("bad root: mbUnpack");
meUnpack(&me, &mb, 0);
if(!deUnpack(&de, &me))
sysfatal("bad root: dirUnpack");
if(!de.qidSpace)
sysfatal("bad root: no qidSpace");
qid = de.qidMax;



On Thu, Dec 17, 2015 at 11:13 PM, Steve Simon  wrote:

> I have long wanted to do this but on the one occasion I tried I got lost
> inside fossil.
>
> the problem is fossil expects venti to have a hierarchy of the form
> /active/ and /archive/ where as your venti does not, I guess it has just
> your home dir.
>
> it is easy to create the required placeholder in a new fossil attached to
> your fossil. what is missing is a fossil admin command to create a new
> directory but attach it to a given, existing venti score. if you could do
> this you could creat an empty fossil attached to your existing fossil and
> then populate your /archive//mmdd/usr/yourname with each venti score
> you have from your old vac(1)s.
>
> have a look at the fossil create command.
>
> -Steve
>
>
>
>
> On 17 Dec 2015, at 17:05, Ole-Hjalmar Kristensen <
> ole.hjalmar.kristen...@gmail.com> wrote:
>
> I have a file server running Linux at home, with a normal ext3 file system
> and a plan9port venti. I use this venti for vac backup of both the ext3
> file system and other Linux boxes. However, the 2 TB ext3 is running out of
> space, while the venti is roughly 50% full. I could just buy a bigger disk,
> of course, but the exit3 itself is mainly an archive (pictures, video, and
> many years of accumulated documents and software), so I consider switching
> to a fossil+venti file server instead. The fossil manual says "The score
> should have been generated by fossil rather than by vac, so that the
> appropriate snapshot metadata is present". Is there any way of coercing
> fossil to initialize itself properly from a score produced by vac? I could
> copy the files from ext3, but would likely run out of space in fossil,
> which I gather is a bad idea. I run a 9front cwfs auth/cpu/file server, but
> have no experience with fossil, so any help is welcome.
>
>
>
>
>
>

Re: [9fans] Musings on Interfaces

[Ole-Hjalmar Kristensen]

I use Sam mostly for the remote editing facility, but I should perhaps try
to add plumber rules to  use a remote 'B' command to trigger opening the
files from the remote machine like James A. Robinson mentioned. Apart from
that, acme is my main command center. I usually have acme straddling two
monitors, with command windows running ssh to Solaris, Linux, and Windows
boxes. The file system is shared, so this lets me have essentially the same
environment on multiple platforms. Also, I find the acme button 3
invaluable when reading debug logs, since clicking on a file:line output in
the log immediately lets me see in which context the output was generated.

On Fri, Sep 2, 2016 at 9:34 AM, Steve Simon  wrote:

> hi,
>
> Sam has been my only editor since the X11 port was released in about 1992.
> I have not really tried acme, I never gave it a real chance but I used to
> use it
> to edit the plan 9 wiki so I have a little skill.
>
> I agree scroll select is the one feature I would add - I have a feeling
> the 9front guys may have already done this...
>
> I think it is just habit but I find Sam so comfortable I just resist
> change.
>
> -Steve
>
>
> On 1 Sep 2016, at 23:56, Winston Kodogo  wrote:
>
> Thanks to Brantley for his thoughtful musings. Me, I love many things
> about Sam, but I just can't use it as my everyday editor. The structural
> regular expression stuff is a work of genius, but I still find, such are my
> limitations, that the user interface is just too clunky and retro.
>
> On 2 September 2016 at 02:42, Brantley Coile  wrote:
>
>> I think I’ve been a member of 9fans for its entire history. The earliest
>> saved 9fans email in my /mail/box/bwc is dated 2001. But most of the time I
>> have not said much. Given that the list isn’t very busy these days, and
>> that I’m doing a lot of thinking about Plan 9, I thought I would post some
>> of my seemingly random musings.
>>
>> Today I’m thinking about Plan 9’s interfaces.
>>
>> The reason for thinking about those is that I’ve just switch back to
>> sam(1) from acme(1). No real reason, except for the old adage, a change is
>> as good as a rest. I’ve been working 10 to 12 hour days, six days a week
>> lately. I just wanted to change things a bit. Nothing against acme. I’ve
>> been using it for many years and it is a great tool.
>>
>> The one time that Ken Thompson visited my office, when I had an office in
>> Redwood City, he noticed that I was using acme and made a comment to the
>> effect that “you are one of those.” He uses sam as do many of the folks who
>> created Plan 9. Many of the original folks also use acme. I had did a poll
>> years ago but can’t seem to find the results. As did I for many years, even
>> after acme make its appearance. I had gotten a version of it working on my
>> Unix using an Teletype 630 terminal, downloading the samterm and all. It
>> was the main Plan 9 editor during my very brief tenure at Bell Labs in
>> 1990. Acme came after I left with the arrival of Phil Winterbottom and his
>> Alef language. The window manager was 8 1/2, which is like rio(1) without
>> the bumpers one can use to move and resize the window.
>>
>> I must say that it is refreshing to be back with the older editor. I did
>> have modify rio to look for an environmental variable that tells it not to
>> do acme chording. I kept trying to use chording in sam and realized that
>> part of the problem was that I could still use it in rio. So, I added a
>> shell variable that turned that feature of rio off. After that subconscious
>> chording stopped.
>>
>> I don’t think that sam is better than acme, or even the other way around.
>> Both do a good job of getting the job done. They are different. And that
>> difference has an affect on the way one used the system. When I use acme, I
>> mostly stay in acme, using the win program for my shell access. It becomes
>> a kind of integrated environment. With sam, I seem to use tools like sed
>> and awk in the rio windows, like sed and awk more than when I was using
>> acme. I had a similar thing happen when in the 1980’s I dropped vi for ed.
>> I used ed until the 1990’s when I was able to switch to sam full time.
>>
>> But my use of edit commands in sam is the biggest difference between it
>> and acme.
>>
>> In sam, I think more about how to modify things using the command window
>> rather than moving the mouse around and clicking on things. The command
>> language in acme using the Edit command is the same, but somehow it feels
>> different. There is something to be said for the convenience of the command
>> windows in sam.
>>
>> If I thought of the change as an experiment, one result would be the time
>> it took me to not have to think about which editor I was using while
>> working. Our tools should be, for the most part, transparent. It took about
>> a week to switch back to sam from acme. That time is certainly a function
>> of how much I used sam in the past.
>>
>> I’m very grateful to still be using these tools

Re: [9fans] Question re source trees

[Ole-Hjalmar Kristensen]

This is great news. I am thinking of using the Pi3 to control a mobile
platform, and having wifi support without using an external wireless router
would simplify the hardware.

On Thu, Sep 29, 2016 at 10:08 AM, Richard Miller <9f...@hamnavoe.com> wrote:

> > I was wondering whether the changes made
> > by Mr. Miller for supporting the Raspberry Pi were getting folded
> > back into a source tree available via means other than those with
> > an active plan9 system (I see references to  /n/sources/contrib/miller
> > which I assume is a 9fs mount).
>
> You can also use a web browser to access /n/sources; it's mapped to
> http://plan9.bell-labs.com/sources (when the server isn't down).
>
> Source for all the raspberry pi kernel changes is kept up to date in
> http://plan9.bell-labs.com/sources/contrib/miller/9/bcm
>
> > ... I couldn't find anything discussing whether other
> > hardware was supported (e.g., the built-in wifi on the Pi 3).
>
> The Broadcom 43430 wifi chip on the pi3 turns out to be functionally
> almost the same as the 4330, for which I had written a driver some
> time ago for another platform.  The client who commissioned that work
> has generously agreed to its release, so 9piwifi support will appear
> shortly.  (The driver is working but WPA authentication needs a few
> tweaks.)
>
> I can also supply bluetooth code for the pi3 if anyone is interested.
>
>
>

Re: [9fans] Maintenance of an auth server files vs a dns+dhcp+tftp server

[Ole-Hjalmar Kristensen]

On Tue, Nov 15, 2016 at 8:05 PM, Stanley Lieber  wrote:

> "James A. Robinson"  wrote:
>
> >So in a canonical installation the auth server mounts its root from the
> >file server?
> >
> >On Tue, Nov 15, 2016 at 10:47 AM Stanley Lieber  wrote:
> >
> >> The idea is that there is one file system shared by all the
> >neighboring
> >> systems. The canonical Plan 9 installation comprises one disk file
> >server
> >> and many diskless computing machines (auth servers, cpu servers,
> >terminals).
> >>
>
> Yes. You can arrange for hands-free booting by storing  the same
> authid/authdom/password in the nvram of both the file server and the auth
> server. I usually boot the auth server from a 9fat partition or a USB key,
> then tcp (actually, tls) mount the root file system from the file server.
>
> sl
>
>
Is this the reason that it is actually possible to boot a combined
auth/cpu/file server at all? I mean, the auth server stores /adm/keys on
the file server, right? And normally you would need to authenticate
yourself to attach to the file server, which would be kind of difficult,
since it is the auth server that is trying to access the key file...

Ole-Hj.

Re: [9fans] Plan 9 5th Edition

[Ole-Hjalmar Kristensen]

I think there was a port of gcc at one time. It should be possible to use
that to port later versions of gcc.
Go is already ported, AFAIK, but I have not yet found an excuse to try it
out.
Personally, I would really like to have  Ada (Gnat) working on Plan9. I
have made enough errors in C and C++ for a lifetime...
If you have gcc, it does not take that much work to get Gnat working for a
new platform. Most of the work would be the interface to the tasking
system, I think.
Mercurial (and Python) already runs on 9Front.

Ole-Hj.


On Wed, Nov 16, 2016 at 11:27 PM, Charlie Lin 
wrote:

> Any plans for Plan 9 5th edition?
>
> My desires:
> ISO-compliant C compiler and preprocessor
> Port other programming languages (especially Go) to here
> Start a source code repository
> Port Git, SVN, Mercurial, et cetera to here
>

Re: [9fans] Do you use fossil or venti under Linux?

[Ole-Hjalmar Kristensen]

Yes, I have been running a venti at home for some years, and a much smaller
one at work. The venti at home has about 2Tb of data and runs on the linux
machine which I uses as a SMB file server, and the file server uses the
venti for backup. In addition, various laptops also use the same venti for
backup. At work, I back up my home directory every night, simply because it
is much less hassle than going through the sysadmins if I need to restore
somthing. Saved my ass many times. I originally ran the venti at work on my
Solaris desktop machine, but had to move it to another (linux) machine
after the constant update of the venti index threatened to destroy my SSD
disk... It sure was fast, though.

I have only played around with fossil a bit, both on linux and on a 9front
machine. My intention was to eventually replace the SMB server with a
fossil server, but I do not like the fact that fossil crashes if you fill
up its write buffer on disk. So I would have to fix that first by making it
block and start a dump to venti if it get close to the capacity.

Another thing with fossil that irks me is that the file formats for fossil
and vac (which I use for backup) seem to have diverged. This is
unfortunate, otherwise I could easily experiment with switching from my SMB
server to a fossil server by initializing the fossil file system from a vac
score. (Yes, I know that I would be missing  /archive, but I could always
make a new fossil command to initialize a directory from a given score
after having initialized a fossil file system)

Snipped from vac/file.c

* Fossil generates slightly different vac files, due to a now
 * impossible-to-change bug, which contain a VtEntry
 * for just one venti file, that itself contains the expected
 * three directory entries.  Sigh.

I cannot see why the bug is impossible to change if I do not need backwards
compatibility, though.

Ole-Hj.

On Tue, Jun 6, 2017 at 11:39 PM, James A. Robinson 
wrote:

> While I was playing around getting v9fs mounts to work, I see that
> plan9port has added fossil and venti at some point.  I was curious whether
> or not anyone was actually running these under Linux?
>
> Jim
>
>

[9fans] push dataflow shell

[Ole-Hjalmar Kristensen]

I am looking at https://code.google.com/archive/p/push/

According to the page, "This is the new unix port of push. It should work
on any unices supported by plan 9 port(a unix Plan 9 compatibility layer,
it's available at http://swtch.com/plan9port/please install it first).",
but I am a bit puzzled, since, when I unpack the archive, the source code
is in limbo. Now, I have nothing against limbo (or Inferno), but I wasn't
aware that plan9port includes a limbo compiler. Does anyone know if there
is a C version of push somewhere? I'm asking here, since I imagine some of
the contributors to push may be reading this list.

Ole-Hj.

Re: [9fans] Change of plan9 partition size

[Ole-Hjalmar Kristensen]

I would be careful with running Venti on a SD card. Venti eats SD cards,
USB sticks and even SSD disks for breakfast, since the index is heavily
modified, and will rot after a while. Your actual data (arenas) will be
fine, though, so you can recover by rebuilding the index somewhere else.
Been there, done that...

Ole-Hj.

On Thu, Oct 12, 2017 at 12:26 PM,  wrote:

> On Thu, Oct 12, 2017 at 11:33 AM, Pavel Klinkovský
>  wrote:
> >> 9pi has about 1,7GB plan9 partition size.
> >> On my 8GB SD card the rest is unused...
> >
> >
> > Maybe it would be wise to use Venti on that unsed space... ;)
> >
> > Pavel
> >
>
> If you don't have a venti server somewhere near your pi yet, then
> definitely yes.
>
>

[9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

Based on copy.c and readlist.c, I have cobbled together a venti client to
copy a list of venti blocks from one venti server to another. I am thinking
of using it to incrementally replicate the contents on one site site to
another. It could even be used for two-way replication, since the CAS and
deduplicating properties of venti ensures that you will never have write
conflicts at a block level.

I have tried it out by feeding it with the output from printarenas, and it
seems to work reasonably well. Does anyone have any good ideas about how to
incrementally extract the set of scores that has been added to a venti
server? You could extract the whole set of scores and do a diff with an old
set of course, but that's rather inefficient.

Ole-Hj.


#include 
#include 
#include 
#include 
#include 

enum
{
// XXX What to do here?
VtMaxLumpSize = 65535,
};

char *srchost;
char *dsthost;
Biobuf b;
VtConn *zsrc;
VtConn *zdst;
uchar *buf;
void run(Biobuf*);
int nn;

void
usage(void)
{
fprint(2, "usage: copylist srchost dsthost list\n");
threadexitsall("usage");
}

int
parsescore(uchar *score, char *buf, int n)
{
int i, c;

memset(score, 0, VtScoreSize);

if(n != VtScoreSize*2){
werrstr("score wrong length %d", n);
return -1;
}
for(i=0; i= '0' && buf[i] <= '9')
c = buf[i] - '0';
else if(buf[i] >= 'a' && buf[i] <= 'f')
c = buf[i] - 'a' + 10;
else if(buf[i] >= 'A' && buf[i] <= 'F')
c = buf[i] - 'A' + 10;
else {
c = buf[i];
werrstr("bad score char %d '%c'", c, c);
return -1;
}

if((i & 1) == 0)
c <<= 4;

score[i>>1] |= c;
}
return 0;
}

void
threadmain(int argc, char *argv[])
{
int fd, i;

ARGBEGIN{
default:
usage();
break;
}ARGEND

if(argc < 2)
usage();

fmtinstall('V', vtscorefmt);
buf = vtmallocz(VtMaxLumpSize);

srchost = argv[0];
zsrc = vtdial(srchost);
if(zsrc == nil)
sysfatal("could not dial src server: %r");
if(vtconnect(zsrc) < 0)
sysfatal("vtconnect src: %r");

dsthost = argv[1];
zdst = vtdial(dsthost);
if(zdst == nil)
sysfatal("could not dial dst server: %r");
if(vtconnect(zdst) < 0)
sysfatal("vtconnect dst: %r");

if(argc == 2){
Binit(&b, 0, OREAD);
run(&b);
}else{
for(i=2; i

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

Hmm. On both my plan9port and on a 9front system I find printarenas.c, but
no script. Maybe you are thinking of the script for backup of individual
arenas to file? Yes, that could be a starting point.

Anyway, printarenas.c doesn't look too scary, basically a loop checking all
(or matching) arenas. It seems possible to modify the logic to start at a
specific offset.

Not running fossil at the moment, btw., my main file server is a Linux box,
but I use vac for backup, both at home and at work. Fossil is definitely on
my todo list, although the reported behavior when running out of space is a
bit scary. Do you know why it does not simply block further requests while
checkpointing to venti, or even better, starts a snapshot before it runs
out of space?

On Tue, Dec 12, 2017 at 3:07 PM, Steve Simon  wrote:

> printarenas is a script - it walks through all your arenas at each offset.
>
> You could craft another script that remembers the last arena and offset
> you successfully
> transferred and only send those after that.
>
> I think there is a pattern where you can save the last arena,offset in the
> local
> fossil. Then you could mount the remote venti to check that last
> arena,offset
> that actually arrived and stuck to the disk on the remote site.
>
> On a similar subject I have 10 years of backups from a decomissioned work
> server
> that I need to merge into my home venti one of these days...
>
> -Steve
>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

Thanks for the tip about mounting with 9fs. I have used vacfs on Linux ,
though.
But why so slow? Did you import a root with lots of backup versions? It was
partly because of that I made this client which can import venti blocks
without needing to traverse a file tree over and over again.

On Tue, Dec 12, 2017 at 4:45 PM, Steven Stallion 
wrote:

> Get ready to wait! It took almost a month for me to import about 30GB
> from a decommissioned file server. It was well worth the wait though -
> if you place the the resulting .vac file under /lib/vac (or
> $home/lib/vac) you can just use 9fs to mount with zero fuss.
>
> On a related note, once sources starting having issues with
> availability, I started running nightly snaps of my contrib directory
> via cron:
>
> contrib=/n/sources/contrib/$user
> 9fs sources
> @{cd $contrib && vac -a $home/lib/vac/contrib.vac .} >[2]/dev/null
>
> Now I have a dump-like history of changes I've made to my contrib
> directory without the need to connect to sources:
>
> % 9fs contrib.vac
> % lc /n/contrib
> 201520162017
>
> Cheers,
> Steve
>
> On Tue, Dec 12, 2017 at 8:07 AM, Steve Simon  wrote:
> > printarenas is a script - it walks through all your arenas at each
> offset.
> >
> > You could craft another script that remembers the last arena and offset
> you successfully
> > transferred and only send those after that.
> >
> > I think there is a pattern where you can save the last arena,offset in
> the local
> > fossil. Then you could mount the remote venti to check that last
> arena,offset
> > that actually arrived and stuck to the disk on the remote site.
> >
> > On a similar subject I have 10 years of backups from a decomissioned
> work server
> > that I need to merge into my home venti one of these days...
> >
> > -Steve
> >
>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

Same place as I found another useful script, dumpvacroots:

#!/bin/rc
# dumpvacroots - dumps all the vac scores ever stored to the venti server
# if nothing else, this illustrates that you have to control access
# to the physical disks storing the archive!

ventihttp=`{
echo $venti | sed 's/^[a-z]+!([0-9\.]+)![a-z0-9]+$/\1/
s/^[a-z]+!([0-9\.]+)/\1/; s/$/:8000/'
}

hget http://$ventihttp/index |
awk '
 /^index=/ { blockSize = 0 + substr($3, 11) }
 /^arena=/ { arena = substr($1, 7) }
 /^arena=/ {
start = (0 + substr($5, 2)) - blockSize
printf("venti/printarena -o %.0f %s\n", start, $3 "")
}
' |
rc |
awk '$3 == 16 { printf("vac:%s\n", $2 "") }'

This definitely looks like it could be hacked to support an incremental
dump of scores.

No printarenas there on my (9front) system, though. I'll have to see on a
proper plan9 system, maybe.

On Tue, Dec 12, 2017 at 8:53 PM, Steve Simon  wrote:

> /sys/src/cmd/venti/words/printarenas
>
> no idea why it lived there though.
>
> -Steve
>
>
> On 12 Dec 2017, at 18:33, Ole-Hjalmar Kristensen <
> ole.hjalmar.kristen...@gmail.com> wrote:
>
> Hmm. On both my plan9port and on a 9front system I find printarenas.c, but
> no script. Maybe you are thinking of the script for backup of individual
> arenas to file? Yes, that could be a starting point.
>
> Anyway, printarenas.c doesn't look too scary, basically a loop checking
> all (or matching) arenas. It seems possible to modify the logic to start at
> a specific offset.
>
> Not running fossil at the moment, btw., my main file server is a Linux
> box, but I use vac for backup, both at home and at work. Fossil is
> definitely on my todo list, although the reported behavior when running out
> of space is a bit scary. Do you know why it does not simply block further
> requests while checkpointing to venti, or even better, starts a snapshot
> before it runs out of space?
>
> On Tue, Dec 12, 2017 at 3:07 PM, Steve Simon  wrote:
>
>> printarenas is a script - it walks through all your arenas at each offset.
>>
>> You could craft another script that remembers the last arena and offset
>> you successfully
>> transferred and only send those after that.
>>
>> I think there is a pattern where you can save the last arena,offset in
>> the local
>> fossil. Then you could mount the remote venti to check that last
>> arena,offset
>> that actually arrived and stuck to the disk on the remote site.
>>
>> On a similar subject I have 10 years of backups from a decomissioned work
>> server
>> that I need to merge into my home venti one of these days...
>>
>> -Steve
>>
>>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

I can understand that it cannot fill up. What I do not understand is why
there are no safeguards in place to ensure that it doesn't. (And my inner
geek wants to know)
As you say, in reality it will not fill up unless you dump huge amounts of
data on it at once. Unfortunately, this is just what I intended to do, dump
a 1.5 TB Linux file system on it. :-)

On Tue, Dec 12, 2017 at 9:15 PM, Steve Simon  wrote:

> Re: fossil
>
> Fossil must not fill up, however I would say that the dropoff was the lack
> of clear
> documentation stating this.
>
> Fossil has two modes of operation.
>
> As a stand alone filesystem, not really intented (I believe) as a
> production
> system, more as a replacement for kfs - for laptops or installation
> systems.
>
> A full fossil system is when it is combined with a local venti (venti on
> the same
> machine or on a fast, low latency network connection). Here most files are
> pulled
> from venti (in the limit fossil only contains a single score which
> redirects the root
> of the filesystem to a venti score. However as you change files the new
> version
> is stored on fossil.
>
> Every night aty 4 or 5 am (by convention) fossil does a snap, bumps it
> epoch which
> marks all the changed files as readonly and further changes creates a new
> file.
> The readonly files are then written to venti in the background and their
> space in fossil
> reclaimed.
>
> This means the fossil only needs to be big enough to contain all the
> changes you
> are likely to make in a day - in reality 10Gb or fossil will never fill up
> unless
> you decide to archive your entire dvd collection on the same day.
> I have been running fossil and venti since 2004. Fossil did have problems
> doing
> ephemerial dumps (short lived dumps every 15 mins which live for a few
> days).
> This bug used to cause occasional fossil crashes but venti never lost a
> byte.
>
> The bug was fixed before the labs froze and fossil has been solid since.
>
> I used an ssd for venti which helps its performance, though even with this
> it will
> never match liniux filesystem performance (cwfs may well do better), but I
> know it
> and its fast enough for me for now.
>
> -Steve
>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

Yes, I know. I was thinking along the same lines a while ago, we even
discussed this here on this mailing list. I did some digging, and I found
this interesting comment in vac/file.c:

/*
 
 *
 * Fossil generates slightly different vac files, due to a now
 * impossible-to-change bug, which contain a VtEntry
 * for just one venti file, that itself contains the expected
 * three directory entries.  Sigh.
 */
VacFile*
_vacfileroot(VacFs *fs, VtFile *r)

Ole-Hj

On Tue, Dec 12, 2017 at 9:38 PM, Steve Simon  wrote:

> The best solution (imho) for what you want to do is the feature I never
> added.
>
> It would be great if you could vac up your linux fs and then just cut and
> past the
> vac score into fossil's console with a command like this:
>
> main import -v 7478923893289ef928932a9888c98b2333 /active/usr/ole/linux
>
> the alternative is a 1.6Tb fossil.
>
> -Steve
>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

Yes, you better have high-endurance SSD's. I put the venti index at work on
an ordinary SSD, and it lasted six months. The log itself was fine, of
course, so I only had to rebuild the index to recover. This was plan9port
on Solaris, btw.
Now this venti runs on an ordinary disk, the speed is less, but not that
much, since I moved it to another machine with about 1G allocated to venti
buffer caches.

On Tue, Dec 12, 2017 at 10:02 PM, Steven Stallion 
wrote:

> I have a similar setup. On my file server I have a mirrored pair of
> high-endurance SSDs tied together via devfs with two fossil file
> systems: main and other. main is a 32GB write cache which is dumped
> each night at midnight (this is similar to the labs configuration for
> sources). other is the remaining 96GB for data that doesn't need to
> survive if both SSDs happen to fail at the same time.
>
> My venti store is run on a large Linux machine (~6TB of RAID6 storage)
> and is served via plan9port. Another highly recommended setup is if
> you happen to have a Coraid EtherDrive (I'm biased towards the SRX
> line) this make fantastic stores via the magic of AoE. Unfortunately I
> don't have the rack space, otherwise I'd be using one of those
> instead.
>
> If you're curious about the venti-on-linux setup, I have some scripts
> and a README posted on sources:
> https://9p.io/magic/webls?dir=/sources/contrib/stallion/venti
>
> Somewhat more recently, I wrote a collectd client for plan9 and I also
> monitor my file server using nagios. If there's any interest, I'd be
> happy to post those sources as well.
>
> Cheers,
> Steve
>
> On Tue, Dec 12, 2017 at 2:15 PM, Steve Simon  wrote:
> > Re: fossil
> >
> > Fossil must not fill up, however I would say that the dropoff was the
> lack of clear
> > documentation stating this.
> >
> > Fossil has two modes of operation.
> >
> > As a stand alone filesystem, not really intented (I believe) as a
> production
> > system, more as a replacement for kfs - for laptops or installation
> systems.
> >
> > A full fossil system is when it is combined with a local venti (venti on
> the same
> > machine or on a fast, low latency network connection). Here most files
> are pulled
> > from venti (in the limit fossil only contains a single score which
> redirects the root
> > of the filesystem to a venti score. However as you change files the new
> version
> > is stored on fossil.
> >
> > Every night aty 4 or 5 am (by convention) fossil does a snap, bumps it
> epoch which
> > marks all the changed files as readonly and further changes creates a
> new file.
> > The readonly files are then written to venti in the background and their
> space in fossil
> > reclaimed.
> >
> > This means the fossil only needs to be big enough to contain all the
> changes you
> > are likely to make in a day - in reality 10Gb or fossil will never fill
> up unless
> > you decide to archive your entire dvd collection on the same day.
> > I have been running fossil and venti since 2004. Fossil did have
> problems doing
> > ephemerial dumps (short lived dumps every 15 mins which live for a few
> days).
> > This bug used to cause occasional fossil crashes but venti never lost a
> byte.
> >
> > The bug was fixed before the labs froze and fossil has been solid since.
> >
> > I used an ssd for venti which helps its performance, though even with
> this it will
> > never match liniux filesystem performance (cwfs may well do better), but
> I know it
> > and its fast enough for me for now.
> >
> > -Steve
> >
>
>

Re: [9fans] two ventis?

[Ole-Hjalmar Kristensen]

Strictly speaking, isn't venti just content-addressable block storage, not
a file system? Anyway, I'm curious to know what you are going to use this
for.

On Tue, Dec 12, 2017 at 9:41 PM, Steve Simon  wrote:

> Can a venti instance be configured to service to two seperate venti
> filesystems? They would need different names and different listner
> names but can they share a process / buffer cache?
>
> I guess the alternative is to run two seperate instances.
>
> -Steve
>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

No need to be sorry. I've been looking at the code now and then, but
haven't really got the hang of the difference between the vac and venti
formats.

On Wed, Dec 13, 2017 at 1:03 AM, Steve Simon  wrote:

> grief, sorry.
>
> what can i say, too old, too many kids. important stuff gets pushed out of
> my brain (against my will) to make room for the lyrics of “Let it go”.
>
>
> On 12 Dec 2017, at 21:40, Ole-Hjalmar Kristensen <
> ole.hjalmar.kristen...@gmail.com> wrote:
>
> Yes, I know. I was thinking along the same lines a while ago, we even
> discussed this here on this mailing list. I did some digging, and I found
> this interesting comment in vac/file.c:
>
> /*
>  
>  *
>  * Fossil generates slightly different vac files, due to a now
>  * impossible-to-change bug, which contain a VtEntry
>  * for just one venti file, that itself contains the expected
>  * three directory entries.  Sigh.
>  */
> VacFile*
> _vacfileroot(VacFs *fs, VtFile *r)
>
> Ole-Hj
>
> On Tue, Dec 12, 2017 at 9:38 PM, Steve Simon  wrote:
>
>> The best solution (imho) for what you want to do is the feature I never
>> added.
>>
>> It would be great if you could vac up your linux fs and then just cut and
>> past the
>> vac score into fossil's console with a command like this:
>>
>> main import -v 7478923893289ef928932a9888c98b2333 /active/usr/ole/linux
>>
>> the alternative is a 1.6Tb fossil.
>>
>> -Steve
>>
>>
>

Re: [9fans] two ventis?

[Ole-Hjalmar Kristensen]

I imagine that you would get problems with duplicates. You could duplicate
enough of the caches and internal structures that it would work, but then
you would be close to having two instances of venti anyway. But maybe, if
you removed the assumption that there are no duplicates, and changed venti
to just index the first instance of a score/value pair.

On Wed, Dec 13, 2017 at 1:08 AM, Steve Simon  wrote:

> true
>
> i have my old work archives and rather than merge that venti into my main
> one i could just mount it via vacfs on demand (from 9fs). but can one venti
> cope with two incompatible sets of arenas?
>
> -Steve
>
>
> On 12 Dec 2017, at 21:58, Ole-Hjalmar Kristensen <
> ole.hjalmar.kristen...@gmail.com> wrote:
>
> Strictly speaking, isn't venti just content-addressable block storage, not
> a file system? Anyway, I'm curious to know what you are going to use this
> for.
>
> On Tue, Dec 12, 2017 at 9:41 PM, Steve Simon  wrote:
>
>> Can a venti instance be configured to service to two seperate venti
>> filesystems? They would need different names and different listner
>> names but can they share a process / buffer cache?
>>
>> I guess the alternative is to run two seperate instances.
>>
>> -Steve
>>
>>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

I don't know either, but when I tried flfmt with a vac score as an
experiment, I got this:

ole@ole-TECRA-R940 ~/Desktop/plan9 $ bin/fossil/flfmt -h 192.168.0.101 -v
f648dbae0075eb73bc394ad6cd4c059e655e127c fossil.dat
fs header block already exists; are you sure? [y/n]: y
fs file is mounted via devmnt (is not a kernel device); are you sure?
[y/n]: y
0xfb1e734c
0x1d1feaf1
c85978546e4048fce83120d3992cfc2f57ff2f8c
bin/fossil/flfmt: bad root: no qidSpace

/*
 * Maximum qid is recorded in root's msource, entry #2 (conveniently in
e).
 */
ventiRead(e.score, VtDataType);
if(!mbUnpack(&mb, buf, bsize))
sysfatal("bad root: mbUnpack");
meUnpack(&me, &mb, 0);
if(!deUnpack(&de, &me))
sysfatal("bad root: dirUnpack");
if(!de.qidSpace)
sysfatal("bad root: no qidSpace");
qid = de.qidMax;

It seems that the vac archive does not contain the max qid that
flfmt needs. This seems strange to me, as vac -a should need this info just
as much as fossil needs it. Maybe it's tucked away somewhere else. Guess I
need to look some more at the code.

Digging further, I found the comment in file.c, but did not pursue the matter:

 * Fossil generates slightly different vac files, due to a now
 * impossible-to-change bug, which contain a VtEntry
 * for just one venti file, that itself contains the expected
 * three directory entries.  Sigh.
 */
VacFile*
_vacfileroot(VacFs *fs, VtFile *r)


On Wed, Dec 13, 2017 at 11:00 AM, Steve Simon  wrote:

> I don't think there is any difference between vac and what fossil uses,
> just where it appears in the hierarchy (though maybe I am wrong).
>
> Fossil adds a fixed upper layer of hierarchy
>
> active
> dump
> 
> 
> snap
> 
> 
> 
>
> The difficulty is how to convince fossil to install a score into its
> hierarchy as though
> its one that it created.
>
> I am pretty sure this is doable, it just needs a rather deep understanding
> of how fossil
> works and when I tried to do it I discovered fossil is really rather
> complex.
>
> -Steve
>
>

Re: [9fans] A potentially useful venti client

[Ole-Hjalmar Kristensen]

Here is a pointer to a discussion on comp.os.plan9, but I did not really
get a clear understanding of whether it was possible or not. It seems to me
that it was possible at some time, but based on my own findings, changes to
the format may have made vac and fossil incompatible.

On Wed, Dec 13, 2017 at 12:22 PM, Richard Miller <9f...@hamnavoe.com> wrote:

> > The difficulty is how to convince fossil to install a score into its
> hierarchy as though
> > its one that it created.
>
> Wouldn't that cause a problem with the two origin file systems
> having overlapping Qid spaces?  I think you would need to walk
> and rebuild the directory tree of the vac being inserted, to
> assign new Qid.path values.
>
>
>

Re: [9fans] Is fossil/venti file system a good choice for SSD?

[Ole-Hjalmar Kristensen]

The problem is the index. It is heavily updated, and I had a Fossil
installation that ate my SSD in about 6 months. The log was OK, so I could
rebuild the index on another disk.

On Sat, Feb 3, 2018 at 10:39 AM, lchg  wrote:

> As I know,  fossil/venti file system is log-structured,  so it may be good
> for flash devices, especially in extending life of flash devices.
>

Re: [9fans] What are you using Plan 9 for?

[Ole-Hjalmar Kristensen]

I cannot really say I am using Plan9 for anything serious, although I have
both Plan9 and 9Front running on a couple of old laptops. I keep them
around mainly to see if I can grok the ideas and maybe steal some of them
:-)

But I run the Plan9port tools on both Linux and Solaris, and occasionally
Inferno on Windows, when I want a sane environment there. Acme is my main
editor these days. The 'everything is text' approach works very well when
developing on multiple paltforms (Solaris, Linux, BSD, Windows). In Acme,
the left button combined with the plumber  is really usefull when jumping
from a debug printout in the log to the source code. I run Vac/Venti on
Linux as my backup system both at home and at work.

If I ever get some spare time, I intend to set up a cron job to replicate
the contents between the two Venti servers.



On Thu, Jun 14, 2018 at 5:53 AM, 刘宇宝  wrote:

> Compared to "not for you", "don't care",  "intend to not be successful", I
> like more the topic of cat-v irc channel on freenode set by aiju:  "fun
> fact: you can use multiple operating systems at the same time".
>
> Certainly Plan 9 can't replace Linux/macOS/BSD/Windows, I'm still curious
> its upper bound for a sensible daily usage,  and the best practice from you
> happy experienced Plan 9 users.
>
> I checked mail headers in this mailing list, seems all use Apple Mail,
> iPhone Mail, WebMail with AJAX, Gmail(a lot), ProtonMail,  these emails
> went through Postfix and Exim servers, probably on Linux.
>
> In great harmony, we use kinds of operating system and kinds of software
> on them.
>
> Regards,
> Yubao Liu
>
> > On Jun 14, 2018, at 10:53 AM, N. S. Montanaro  wrote:
> >
> > I think a lot of people discover Plan 9 and want it to be something it
> isn’t, rather than stumble upon it out of necessity. As the FQA says, “Plan
> 9 is not for you."
>
>

Re: [9fans] PDP11 (Was: Re: what heavy negativity!)

[Ole-Hjalmar Kristensen]

I'm not going to argue with someone who has got his hands dirty by actually
doing this but I don't really get this about the tyranny of 9p. Isn't the
point of the tag field to identify the request? What is stopping the client
from issuing multiple requests and match the replies based on the tag? From
the manual:

Each T-message has a tag field, chosen and used by the
  client to identify the message.  The reply to the message
  will have the same tag.  Clients must arrange that no two
  outstanding messages on the same connection have the same
  tag.  An exception is the tag NOTAG, defined as (ushort)~0
  in : the client can use it, when establishing a
  connection, to override tag matching in version messages.



Den ons. 10. okt. 2018, 23.56 skrev Steven Stallion :

> As the guy who wrote the majority of the code that pushed those 1M 4K
> random IOPS erik mentioned, this thread annoys the shit out of me. You
> don't get an award for writing a driver. In fact, it's probably better
> not to be known at all considering the bloody murder one has to commit
> to marry hardware and software together.
>
> Let's be frank, the I/O handling in the kernel is anachronistic. To
> hit those rates, I had to add support for asynchronous and vectored
> I/O not to mention a sizable bit of work by a co-worker to properly
> handle NUMA on our appliances to hit those speeds. As I recall, we had
> to rewrite the scheduler and re-implement locking, which even Charles
> Forsyth had a hand in. Had we the time and resources to implement
> something like zero-copy we'd have done it in a heartbeat.
>
> In the end, it doesn't matter how "fast" a storage driver is in Plan 9
> - as soon as you put a 9P-based filesystem on it, it's going to be
> limited to a single outstanding operation. This is the tyranny of 9P.
> We (Coraid) got around this by avoiding filesystems altogether.
>
> Go solve that problem first.
> On Wed, Oct 10, 2018 at 12:36 PM  wrote:
> >
> > > But the reason I want this is to reduce latency to the first
> > > access, especially for very large files. With read() I have
> > > to wait until the read completes. With mmap() processing can
> > > start much earlier and can be interleaved with background
> > > data fetch or prefetch. With read() a lot more resources
> > > are tied down. If I need random access and don't need to
> > > read all of the data, the application has to do pread(),
> > > pwrite() a lot thus complicating it. With mmap() I can just
> > > map in the whole file and excess reading (beyond what the
> > > app needs) will not be a large fraction.
> >
> > you think doing single 4K page sized reads in the pagefault
> > handler is better than doing precise >4K reads from your
> > application? possibly in a background thread so you can
> > overlap processing with data fetching?
> >
> > the advantage of mmap is not prefetch. its about not to do
> > any I/O when data is already in the *SHARED* buffer cache!
> > which plan9 does not have (except the mntcache, but that is
> > optional and only works for the disk fileservers that maintain
> > ther file qid ver info consistently). its *IS* really a linux
> > thing where all block device i/o goes thru the buffer cache.
> >
> > --
> > cinap
> >
>
>

Re: [9fans] PDP11 (Was: Re: what heavy negativity!)

[Ole-Hjalmar Kristensen]

OK, that makes sense. So it would not stop a client from for example first
read an index block in a B-tree, wait for the result, and then issue read
operations for all the data blocks in parallel. That's exactly the same as
any asynchronous disk subsystem I am acquainted with. Reordering is the
norm.

On Sun, Oct 14, 2018 at 1:21 PM hiro <23h...@gmail.com> wrote:

> there's no tyranny involved.
>
> a client that is fine with the *responses* coming in reordered could
> remember the tag obviously and do whatever you imagine.
>
> the problem is potential reordering of the messages in the kernel
> before responding, even if the 9p transport has guaranteed ordering.
>
> On 10/14/18, Ole-Hjalmar Kristensen 
> wrote:
> > I'm not going to argue with someone who has got his hands dirty by
> actually
> > doing this but I don't really get this about the tyranny of 9p. Isn't the
> > point of the tag field to identify the request? What is stopping the
> client
> > from issuing multiple requests and match the replies based on the tag?
> From
> > the manual:
> >
> > Each T-message has a tag field, chosen and used by the
> >   client to identify the message.  The reply to the message
> >   will have the same tag.  Clients must arrange that no two
> >   outstanding messages on the same connection have the same
> >   tag.  An exception is the tag NOTAG, defined as (ushort)~0
> >   in : the client can use it, when establishing a
> >   connection, to override tag matching in version messages.
> >
> >
> >
> > Den ons. 10. okt. 2018, 23.56 skrev Steven Stallion  >:
> >
> >> As the guy who wrote the majority of the code that pushed those 1M 4K
> >> random IOPS erik mentioned, this thread annoys the shit out of me. You
> >> don't get an award for writing a driver. In fact, it's probably better
> >> not to be known at all considering the bloody murder one has to commit
> >> to marry hardware and software together.
> >>
> >> Let's be frank, the I/O handling in the kernel is anachronistic. To
> >> hit those rates, I had to add support for asynchronous and vectored
> >> I/O not to mention a sizable bit of work by a co-worker to properly
> >> handle NUMA on our appliances to hit those speeds. As I recall, we had
> >> to rewrite the scheduler and re-implement locking, which even Charles
> >> Forsyth had a hand in. Had we the time and resources to implement
> >> something like zero-copy we'd have done it in a heartbeat.
> >>
> >> In the end, it doesn't matter how "fast" a storage driver is in Plan 9
> >> - as soon as you put a 9P-based filesystem on it, it's going to be
> >> limited to a single outstanding operation. This is the tyranny of 9P.
> >> We (Coraid) got around this by avoiding filesystems altogether.
> >>
> >> Go solve that problem first.
> >> On Wed, Oct 10, 2018 at 12:36 PM  wrote:
> >> >
> >> > > But the reason I want this is to reduce latency to the first
> >> > > access, especially for very large files. With read() I have
> >> > > to wait until the read completes. With mmap() processing can
> >> > > start much earlier and can be interleaved with background
> >> > > data fetch or prefetch. With read() a lot more resources
> >> > > are tied down. If I need random access and don't need to
> >> > > read all of the data, the application has to do pread(),
> >> > > pwrite() a lot thus complicating it. With mmap() I can just
> >> > > map in the whole file and excess reading (beyond what the
> >> > > app needs) will not be a large fraction.
> >> >
> >> > you think doing single 4K page sized reads in the pagefault
> >> > handler is better than doing precise >4K reads from your
> >> > application? possibly in a background thread so you can
> >> > overlap processing with data fetching?
> >> >
> >> > the advantage of mmap is not prefetch. its about not to do
> >> > any I/O when data is already in the *SHARED* buffer cache!
> >> > which plan9 does not have (except the mntcache, but that is
> >> > optional and only works for the disk fileservers that maintain
> >> > ther file qid ver info consistently). its *IS* really a linux
> >> > thing where all block device i/o goes thru the buffer cache.
> >> >
> >> > --
> >> > cinap
> >> >
> >>
> >>
> >
>
>

Re: [9fans] Trying to make 9front work on QWERTZ

[Ole-Hjalmar Kristensen]

Can't give a definitive answer, but it works fine on my Norwegian keyboard
which also has a rather different layout from the US keyboard. Unless the
key code is simply not handled, I can't imagine why you get nothing at all.

ons. 24. jul. 2019, 21.03 skrev Jens K. Loewe :

> Ahoy,
>
> I've been trying to give 9front a test ride for a while now, and I'm
> stuck with one specific problem.
>
> So I have a German keyboard layout where <, > and | are on the same
> key. However, while I have no problems with these keys, in 9front the
> key seems to be dead on all of my keyboards. I tried quite a lot of
> them, both inside QEMU on two different computers and on a dedicated
> ThinkPad. Also, using the de layout does not fix that.
>
> Is that a known problem or a configuration error?
>
>

Re: [9fans] question re acme and plumber

[Ole-Hjalmar Kristensen]

I think the text that is selected and sent by button 3 is hard-coded in
acme, so the square brackets acts as delimiters. If you click on the text
to the left of the brackets, the plumber will not see the brackets or
what's inside them. If you sweep and select yourself, the whole selection
goes to the plumber.

tor. 11. jul. 2019, 23.30 skrev James A. Robinson :

> Well, I can see this is getting called:
>
> look.c:187: if(m->ndata plumbsendtofid(plumbsendfid, m) >= 0){
>
> and the m->data is the full line, including the trailing
> ".java:[,]" data.
>
> So it's certainly appears to be sending the data to the plumber.
> But I don't get the same behavior from acme as when I send the
> same text I see in m->data to plumb(1) directly on the command
> line.
>
> Jim
>
>

Re: [9fans] iOS drawterm

[Ole-Hjalmar Kristensen]

I think I agree. Besides, drawterm isn't that bad even over high-latency
VPN. I experimented a bit by running drawterm at work against a plan9
server at home, and it was quite usable, and much better than Emacs running
over X using the same connection. Of course, Emacs IS notoriously bad at
this...

On Wed, Mar 25, 2020 at 6:28 PM Anthony Sorace  wrote:

> VNC is great for what it is, and I certainly wouldn’t object to seeing
> vncs upgraded, but it is not a replacement for drawterm. It does not expose
> local devices in a plan 9 friendly way. In addition to just using drawterm
> as a straightforward terminal, an iOS version would be a very good platform
> for playing around with exposing other capabilities that the device has to
> plan 9. I played around with this a little bit with the original port. VNC
> buys us none of this.
>
> On Mar 25, 2020, at 04:21, Kim Lassila  wrote:
>
> 
>
> On Mar 25, 2020, at 8:19 AM, Anthony Sorace  wrote:
>
> With iOS getting first-class mouse pointer support, I’m looking at the iOS
> drawterm port again. Has anyone touched this since the old GSoC project bit
> rotted out?
>
>
> Drawterm is quite slow at reading and writing pixels on the screen. I
> learned this when I started recording screen in Plan 9 (
> https://github.com/9d0/screencast).
>
> Instead of porting drawterm to different platforms I would like to see
> vncs improved to support the latest version of the Remote Framebuffer
> Protocol (RFC 6143). This would allow a standard VNC client to connect to a
> Plan 9 terminal, support screen resizing, local mouse cursor, and deliver
> all key strokes and mouse chords accurately. VNC is optimized to work over
> a large variety of different networks including high latency links and it
> will therefore offer a better user experience than drawterm, especially
> over wireless.
>
> Kim
>
> *9fans * / 9fans / see discussions
>  + participants
>  + delivery options
>  Permalink
> 
>

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T69dec3540d033863-M15caebf66eb1efe4c2916326
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] plan9port fossil

[Ole-Hjalmar Kristensen]

I am curious about the problems you have with your Linux Mint. I have been
running a relatively large (6 TB max) venti on an old Toshiba laptop with
Intel core 2 processor running Bunsenlabs (Debian) the last couple of
years. It was migrated from a 2 TB venti running on an old Pentium 4 box
with 1 GB running Crunchbang (Debian), when that one ran out of space. Both
installations have been used for backup of various other systems, including
my Linux Mint laptop, my Linux file server, and a 9front installation, and
I have experienced no problems at all with respect to throughput. I have a
fairly sub-optimal config with everything on a single disk:

index sdc-6tb
arenas /dev/sdc1
isect /dev/sdc2
bloom /dev/sdc3
mem 100M
bcmem 200M
icmem 400M

What kind of write speed are you seeing?


On Thu, Mar 19, 2020 at 1:34 PM Lucio De Re  wrote:

> On 3/18/20, Steven Stallion  wrote:
> [ ...]
> >
> > I've had a lot of luck using venti from plan9port with fossil running
> > natively on my plan9 fileserver. I keep a directory on sources (now
> > 9p.io) with some notes and example scripts on how to make this work:
> > http://9p.io/sources/contrib/stallion/venti/. The biggest benefit to
> > this configuration is it makes offsite backups a breeze from the Linux
> > host.
> >
> Nice stuff, Steven. I found my small Linuxmint workstation not up to
> the task, the worst symptom being that shutting venti down takes a
> very long time, tens of minutes, I think. It didn't matter when the
> host was on all the time, but of late power blackouts have made that
> untenable.
> 
> What I wish to contribute here is that using an external drive and
> configuring it as a raw image allows it to be used (I presume even
> shared) between Venti hosts. My most recent (and pretty old)
> configuration is this:
> 
> $ cat sdb.conf
> index   main
> 
> isect   /dev/sdb3:0k-20774910k
> arenas  /dev/sdb3:20774911k-418906111k
> bloom   /dev/sdb3:418906112k-419430400k
> 
> mem 80m
> bcmem   160m
> icmem   256m
> 
> addrtcp!*!venti
> httpaddrtcp!*!8008
> 
> Sadly, there is at least one damaged block and I did not have the
> foresight to set the drive up as a mirror or better.  It is not
> critical, but that would be helpful.
> 
> The equally low priority problem I mentioned in the past: vacfs on p9p
> truncates all large files on reading. Cinap suggested checking for
> mixed-size pointer/integer types, but that becomes a mission. Still,
> it is worth doing.
> 
> The native version of vacfs works flawlessly and mounts quite
> successfully under p9p.
> 
> Lucio.

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T5df913730c26a8d5-Mb6b98cac2c746896e680e858
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[9fans] Excessive index read/write in venti

[Ole-Hjalmar Kristensen]

I run a venti from plan9port under Linux at home. Sometimes, typically
after a massive insert, I notice that venti reads and writes about 5 Mb/s
on the index disk continuously. This continues long after the insert has
finished, and there is a marked performance drop. I have never had the
patience to wait more than 24 hours, but it seems to never stop. If I kill
the venti and restart it, the problem goes away until I do massive inserts
again. After the initial indexing, there is no index activity, and
everything is back to normal. Has anyone observed similar behavior? Known
bugs?

Ole-Hj.

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T549c5d2fdcb79f42-Mc56f84ced7de94502b13f6ad
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[9fans] Venti on OpenBSD - some information and a question

[Ole-Hjalmar Kristensen]

I am in the process of moving a venti from Linux to OpenBSD.
First, unless int _p9dir(struct stat *lst, struct stat *st, char *name, Dir
*d, char **str, char *estr) is patched, it will always return size 0 for
raw partitions.
We need to allow character devices as well, not just block devices:

diff --git a/src/lib9/_p9dir.c b/src/lib9/_p9dir.c
index 58c63573..0fb3410e 100644
--- a/src/lib9/_p9dir.c
+++ b/src/lib9/_p9dir.c
@@ -230,7 +230,7 @@ _p9dir(struct stat *lst, struct stat *st, char *name,
Dir *d, char **str, char *
  d->qid.path = ('c'<<16)|st->st_rdev;
  }
  /* fetch real size for disks */
- if(S_ISBLK(lst->st_mode)){
+ if(S_ISBLK(lst->st_mode) || S_ISCHR(lst->st_mode)){
  if((fd = open(name, O_RDONLY)) >= 0){
  d->length = disksize(fd, st);
  close(fd);
dell-openbsd$ git log | head
commit c1c1b5267fd5e14be531a4b22ed0124b35d427cb
Author: sean 
Date:   Wed Apr 29 11:21:35 2020 +0100

Second, if your raw partitions are MBR partitions, not OpenBSD disk labels,
you will still get 0 size, so venti will not run.
Disk with only MBR partitions:

dell-openbsd# disklabel sd1
16 partitions:
#size   offset  fstype [fsize bsize   cpg]
  c:4883786450  unused
  i:46080  256 unknown
  j: 2560460800256 unknown
  k:  1834980486400770 unknown

In this case you will need to run disklabel and edit the label to get BSD
partitions:
dell-openbsd# disklabel sd1.
16 partitions:
#size   offset  fstype [fsize bsize   cpg]
  c:4883786450  unused
  d:46080  256  4.2BSD   4096 32768 1
  e: 2560460800256  4.2BSD   4096 32768 1
  f:  1834980486400770  4.2BSD   4096 32768 1
  i:46080  256 unknown
  j: 2560460800256 unknown
  k:  1834980486400770 unknown

Now for the question. For some reason, the venti running on OpenBSD is
refusing connections, both on port 80 and on 17034. No error messages from
venti that I can see.
dell-openbsd# bin/venti/venti -d -c /home/ole/venti/v2.conf
2020/0504 21:48:08 venti: conf...httpd tcp!*!80...init...icache
1,895,825,408 bytes = 18,850,537 entries; 16 scache
sync...announce tcp!*!17034...serving.

I can connect to the venti from localhost, but not from any other machine.
However, if I run nc -l on ports 17034 and 80, I can connect from any
machine. It is definitely not the packet filter, since the problem persists
even if I disable the packet filter. Any suggestions about what might be
blocking the connection?

The only real clue I have is that even on localhost I get "connection
refused", but telnet switches to IPv6 and that attempt is accepted:
dell-openbsd$ telnet localhost 17034
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
Trying ::1...
Connected to localhost.
Escape character is '^]'.
venti-04:02-libventi

Is it possible that venti somehow is listening only on the IPv6 address?

Ole-Hj.

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T2c2522d6acc96fdd-Mbdd6fb37a7a064cff1868091
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] Venti on OpenBSD - some information and a question

[Ole-Hjalmar Kristensen]

Thanks, I found the problem by inspection of src/lib9/_p9dialparse.c.
IPv6 is indeed the default when using an address of tcp!*!17034
 /* translate host */
if(strcmp(host, "*") == 0){
ss->ss_family = AF_INET6;
((struct sockaddr_in6*)ss)->sin6_addr = in6addr_any;
}else if((he = gethostbyname(host)) != nil && he->h_addr_list[0] != nil){

So, setting the address string to tcp!dell-openbsd!17034 fixed the problem.
My Linux machines actually have an IPv6 address, the OpenBSD box does not,
so that explains the difference.

Ole-Hj.

On Mon, May 4, 2020 at 10:09 PM  wrote:

> > I can connect to the venti from localhost, but not from any other
> machine.
> > However, if I run nc -l on ports 17034 and 80, I can connect from any
> > machine. It is definitely not the packet filter, since the problem
> persists
> > even if I disable the packet filter. Any suggestions about what might be
> > blocking the connection?
>
> My first guess is that it's listening on the wrong interface.
> Run it under 'ktrace', then run 'kdump' and look for the bind()
> calls and friends.
>
>

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T2c2522d6acc96fdd-M4397dad761b09ff3e7cd76ac
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] Dual dialing/forking sessions to increase 9P throughput

[Ole-Hjalmar Kristensen]

I agree with you that using the existing tag mechanism to keep multiple
requests in flight should be sufficient. I get the impression that this is
not readily supported by the higher level libraries, though.

As an aside, I seem to remember that John Floren sugegsed (and implemented)
changes to the 9P protocol making it more suitable for streaming media by
creating another TCP connection on the side.

On Mon, Jan 4, 2021 at 12:52 AM Ethan Gardener  wrote:

> > The idea, basically, is to use an open flag (OJUMBO) to signal that two
> > connections to the same server should be attempted.
>
> What's the advantage over fcp(1)? 9p can have numerous requests "in
> flight" at once to work around latency issues, but of all the user
> programs, fcp is probably the only one which takes advantage of this.
>
> > If a second
> > connection can be established, it is used for normal 9P transactions,
> > while the first connection is used for large ("jumbo") writes.
> 
> How large is "jumbo"? I believe all the user programs have 8KB buffers at
> present; are you going to change them all?
> 
> I'm not negative about this; just raising the points.

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/Te69bb0fce0f0ffaf-M32ea06de21be8e55f94d0b3b
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] plan9port: acme remoting

[Ole-Hjalmar Kristensen]

Very interesting. In the past, I have resorted to using a combination of
Sam/9term/plumber in similar scenarios, but I really prefer Acme. At the
moment I'm running Acme over X11, since I'm constrained to using Windows on
the desktop, and I've got a working X server, but no Acme on Windows.
Anyone got plan9port to work under Cygwin?

On Mon, Jan 4, 2021 at 5:41 AM marius a. eriksen  wrote:

> Lately, I have needed to do a great deal of my work on remote servers.
> It's been difficult to do this with acme. Previously, I have attempted to
> run acme entirely remotely, and then attach to a local devdraw instance
> . This works okay, but also
> has many drawbacks, among which: (1) it's not economical with networking,
> and so not great for high latency or otherwise poor connections; (2) it's
> not resilient to connection disconnects; (3) often I want access to local
> resources too.
>
> I built an experimental remoting layer in acme to see how far I could get
> in recreating a modern "remoting" experience with acme. Since acme itself,
> of course, is designed around a bunch of 9p servers, this turned out to be
> fairly easy to do, and the resulting experience is nearly transparent.
>
> A remote may be attached to one or more path prefixes. These are similar
> to mount points in a file system. Thus, if the prefix /home/meriksen is
> attached to the remote "dev", then any file with this prefix is transparely
> fetched from the dev. Similarly, when commands are run from any directory
> with this prefix, the command is transparently run on the remote. The
> command is run in an environment that includes the acme and plumber 9p
> servers (forwarded from the local host), and so even acme programs just
> work. For example, if you run 'win' from a directory that is attached to a
> remote, 'win' is run on the remote host, but it accesses the acme 9p file
> tree (which is forwarded from the local acme) and creates its window and
> interacts with acme. It "just works". (I regularly run other programs like
> this too, e.g., acme-lsp, which uses the plumber to coordinate interaction).
>
> The design of the feature is quite simple: a new program, acmesrv
>  is
> run by acme (through ssh) on the remote host. Acme becomes a client of
> acmesrv, through which all further interaction is facilitated. Acmesrv
> itself is really just a 9p multiplexer: the local acme program exports the
> acme and plumber 9p servers; acmesrv provides two new 9p servers (exportfs
> and cmdfs) that gives the local acme access to the remote's file system
> (exportfs) and to run commands (cmdfs).
>
> The local acme manages a session for each remote, and properly deals with
> session disconnect/reconnect, etc. Thus, poor network conditions and
> disconnects are easily supported. Since acme's file handling is anyway
> stateless (the file is not kept open during editing), file state is easily
> maintained through disconnect-reconnect cycles.
>
> There are some synchronous code paths in acme, and these may cause
> temporary freezes when using remoting. For example, when acme queries for
> the existence of a file (stat), this blocks UI updates in acme. On session
> establishment (e.g., the first time a file is accessed on a remote), this
> can cause additional delays, since the file cannot be queried before first
> establishing a new session. I've not found this to be a big deal.
>
> This has been my "daily driver" for about a month at this point, and
> overall I'm very happy with the experience. It creates a nearly transparent
> experience editing remote files, and retains most of acme's functionality
> in remote settings as well. I never hesitate to shut my laptop lid while in
> the middle of a project. Most things just work. There are wrinkles, of
> course, and there are some sharp edges in this (experimental)
> implementation, but, to me, this provides really very useful functionality
> that lets me use acme in more challenging environments. (Not to mention
> that this means I can use a beefy remote host for heavy lifting.)
>
> In case this would be useful to anyone else, this version of acme resides
> in my plan9port tree on github
> .
> (Currently, these diffs: 1
> ,
> 2
> ,
> 3
> ,
> 4
> 

Re: [9fans] Codebase navigation and using tags files in acme

[Ole-Hjalmar Kristensen]

On linux, you can run ctags -x and postprocess the file to append the line
number to the file name instead of having i as a separate field. That way,
you can locate the symbol in the tags file, and right-click on the
file:linenumber.

Also, on linux, we have acme-lsp, which in principle works with any LSP
server. I have tested it with go and C/C++.

On Tue, Aug 17, 2021 at 10:22 PM Ben Hancock  wrote:

> Hello 9fans,
> 
> I've just recently started using the acme editor and am really enjoying
> it, and trying to get the hang of the "acme way" of doing things. One
> bit of functionality that I'm familiar with from other editors is the
> ability to easily look up a function or symbol definition within a
> codebase. In Emacs and vi, this is done by generating tags files (etags
> or ctags), which those editors can parse and allow you to easily jump to
> a definition of the symbol under the point/cursor.
> 
> What's the preferred method or workflow for achieving this in acme? I
> have tried passing a selected symbol to 'g -n' in the window's tag,
> using the Mouse-2 + Mouse-1 chord. That gets me part of the way there
> but isn't effective if the file where the symbol is defined happens to
> be in another directory. I feel like I'm missing something.
> 
> Many thanks!
> 
> - Ben
> 
> --
> Ben Hancock
> www.benghancock.com

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/Tf8ceac12df9da674-Mb6807aa0e9520cbbc2e097ad
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] Codebase navigation and using tags files in acme

[Ole-Hjalmar Kristensen]

I use clangd. Works very well in general. Using it is simple, you launch
clangd via acme-lsp like this:
acme-lsp -server '(\.h)|(\.cc)$:/home/ole/bin/clangd_10.0.0/bin/clangd
--clang-tidy --background-index --cross-file-rename' -workspaces
/home/ole/xcomtest

On Thu, Aug 19, 2021 at 5:51 AM <6o205z...@sneakemail.com> wrote:

> I had never heard of acme-lsp and LSP (except as a Microsoft internal
> thing) until gabi mentioned it earlier in the thread.  I'm interesting in
> playing with acme-lsp for C++.  Which LSP server do you use for C/C++ (I
> see several listed at https://langserver.org/)?
>
> thanks,
> Peter
>
> On 2021-08-18 02:26, Ole-Hjalmar Kristensen
> ole.hjalmar.kristensen-at-gmail.com |9fans| wrote:
>
> On linux, you can run ctags -x and postprocess the file to append the line
> number to the file name instead of having i as a separate field. That way,
> you can locate the symbol in the tags file, and right-click on the
> file:linenumber.
>
> Also, on linux, we have acme-lsp, which in principle works with any LSP
> server. I have tested it with go and C/C++.
>
> On Tue, Aug 17, 2021 at 10:22 PM Ben Hancock  wrote:
>
>> Hello 9fans,
>> 
>> I've just recently started using the acme editor and am really enjoying
>> it, and trying to get the hang of the "acme way" of doing things. One
>> bit of functionality that I'm familiar with from other editors is the
>> ability to easily look up a function or symbol definition within a
>> codebase. In Emacs and vi, this is done by generating tags files (etags
>> or ctags), which those editors can parse and allow you to easily jump to
>> a definition of the symbol under the point/cursor.
>> 
>> What's the preferred method or workflow for achieving this in acme? I
>> have tried passing a selected symbol to 'g -n' in the window's tag,
>> using the Mouse-2 + Mouse-1 chord. That gets me part of the way there
>> but isn't effective if the file where the symbol is defined happens to
>> be in another directory. I feel like I'm missing something.
>> 
>> Many thanks!
>> 
>> - Ben
>> 
>> --
>> Ben Hancock
>> www.benghancock.com
>
> *9fans <https://9fans.topicbox.com/latest>* / 9fans / see discussions
> <https://9fans.topicbox.com/groups/9fans> + participants
> <https://9fans.topicbox.com/groups/9fans/members> + delivery options
> <https://9fans.topicbox.com/groups/9fans/subscription> Permalink
> <https://9fans.topicbox.com/groups/9fans/Tf8ceac12df9da674-M652caba65026de9632210a05>
>

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/Tf8ceac12df9da674-M15ec62c316baf8b49043cae2
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

[9fans] The development environment from Hell

[Ole-Hjalmar Kristensen]

Not exactly Hell, but rather close:
ISPOrganizationLatitudeLongitude
NTE Marked AS Not Available 63.4333 10.6833
😀

This is the tale of a convoluted development environment, not specifically
Plan9, but Plan9port, sam, and acme.

I am working on a largish system, with about 5M LOC. The setup is as
follows. I have a Windows laptop. It needs to be Windows, because of the
office automation applications. It is also the only machine I have at home
which can connect to the company VPN. The office desktop is a nice modern
AMD Ryzen 9 16-core processor with plenty of RAM running Ubuntu. The system
cannot (for reasons beyond my control) be compiled on the Ubuntu, it needs
specifically CentOS 7, so we run a Docker image with CentOS 7 on the Ubuntu
machine. Compiling on the laptop is a no-go anyway, it is much too slow, so
I have not bothered installing WSL and Docker on it.

I started out running VSCode, which is available both on Windows and
Ubuntu, and which has nice remote editing capabilities a la sam, and tons
of plugins. It is a better experience then Visual Studio, in my opinion. It
has a clangd plugin which can be used to navigate the code, but for some
reason clangd chokes on some submodules. acme with acme-lsp gives me the
same navigation capabilities, and I prefer the user interface of acme, so I
have reverted to acme on the Ubuntu machine. Works great, I use acme-lsp on
the modules clangd manages to compile, and ag (parallel grep-like tool) on
the rest. So in the office, everything is hunky-dory. I connect to the
Docker image from acme by running win docker exec, and since both Docker
and the Ubuntu machine has the same view of the files, everything works
seamlessly.

When working from home, acme is not an ideal solution, since the VPN
network has high latency and relatively low bandwith (a fraction of my
500Mbit fiber connection). I run an X server on Windows, and start acme on
the Ubuntu machine. Editing files works fairly well, but commands which
spew large amounts of text really bogs acme down. I could switch to VSCode,
but I would prefer not to.

Now, I have experimented with other solutions which could give me a better
experience. Running sshfs and a local acme would be one possibility, but it
is not really fast enough for searching through the code. (I can search the
5M lines code base in about 2 seconds with ag on the desktop machine).
Copying the whole system to the local machine and do the editing locally,
then run Unison to synchronize, is another possibility, but not ideal.

So I thought about sam, which I also like, although I have not used it as
much as acme. Sam has remote editing, which solves the latency and
bandwidth problem, but I prefer the acme right-click to navigate
compilation errors and grep results. So, can we combine acme and sam in a
meaningful way? It turns out we can. From a local acme, I connect to the
remote Docker in one or more acme windows. This is for compilation and
grep. By adding plumber rules which sends any file which cannot be found
locally to the remotely connected sam, I can right-click on errors and grep
results and get sam to go there. The only thing is that to avoid recursion
in the plumber, I must specifically route the message to the sam named pipe
instead of to the edit port, which means I need to start sam before the
plumber. Also, I have not investigated how I can access the remote clangd
from my local acme.

What do you think? Suggestions?

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T73fbb7a533aef743-Me2086e4438c7fc5148074af1
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] The development environment from Hell

[Ole-Hjalmar Kristensen]

Seems to work very well. Excellent! There is one thing that is a bit
strange, though. Unless I explicitly list my home directory as one of the
remote paths, acme simply does not connect to the remote machine at all. No
big deal, though.
Another slight issue is that because acme loads whole files, opening a huge
file may take quite a bit of time. Sam avoids this issue by only loading
the file on the remote, and transfers only what is needed to refresh the
display. But that's just a different tradeoff. All in all I think I prefer
your acme, especially since it can run acme-lsp on the remote.

On Wed, Jan 12, 2022 at 5:06 AM marius a. eriksen  wrote:

> > So I thought about sam, which I also like, although I have not used it
> as much as acme. Sam has remote editing, which solves the latency and
> bandwidth problem, but I prefer the acme right-click to navigate
> compilation errors and grep results. So, can we combine acme and sam in a
> meaningful way? It turns out we can. From a local acme, I connect to the
> remote Docker in one or more acme windows. This is for compilation and
> grep. By adding plumber rules which sends any file which cannot be found
> locally to the remotely connected sam, I can right-click on errors and grep
> results and get sam to go there. The only thing is that to avoid recursion
> in the plumber, I must specifically route the message to the sam named pipe
> instead of to the edit port, which means I need to start sam before the
> plumber. Also, I have not investigated how I can access the remote clangd
> from my local acme.
>
> An alternative is this patch
> <https://gist.github.com/mariusae/a7b13730b7c5aa08f32b30a64f31856b> which
> implements a proper remoting layer for Acme. (Very similar to the way Sam
> does it: it starts a server over ssh, and then ferries 9p back and forth.)
>
> I use this as my daily driver, and it's _very_ close to a local experience.
>
> See this thread
> <https://www.mail-archive.com/9fans@9fans.net/msg39249.html> for some
> more details.
>
>
>
>
> On Thu, Oct 21, 2021 at 11:16 AM Ole-Hjalmar Kristensen <
> ole.hjalmar.kristen...@gmail.com> wrote:
>
>> Not exactly Hell, but rather close:
>> ISPOrganizationLatitudeLongitude
>> NTE Marked AS Not Available 63.4333 10.6833
>> 😀
>>
>> This is the tale of a convoluted development environment, not
>> specifically Plan9, but Plan9port, sam, and acme.
>>
>> I am working on a largish system, with about 5M LOC. The setup is as
>> follows. I have a Windows laptop. It needs to be Windows, because of the
>> office automation applications. It is also the only machine I have at home
>> which can connect to the company VPN. The office desktop is a nice modern
>> AMD Ryzen 9 16-core processor with plenty of RAM running Ubuntu. The system
>> cannot (for reasons beyond my control) be compiled on the Ubuntu, it needs
>> specifically CentOS 7, so we run a Docker image with CentOS 7 on the Ubuntu
>> machine. Compiling on the laptop is a no-go anyway, it is much too slow, so
>> I have not bothered installing WSL and Docker on it.
>>
>> I started out running VSCode, which is available both on Windows and
>> Ubuntu, and which has nice remote editing capabilities a la sam, and tons
>> of plugins. It is a better experience then Visual Studio, in my opinion. It
>> has a clangd plugin which can be used to navigate the code, but for some
>> reason clangd chokes on some submodules. acme with acme-lsp gives me the
>> same navigation capabilities, and I prefer the user interface of acme, so I
>> have reverted to acme on the Ubuntu machine. Works great, I use acme-lsp on
>> the modules clangd manages to compile, and ag (parallel grep-like tool) on
>> the rest. So in the office, everything is hunky-dory. I connect to the
>> Docker image from acme by running win docker exec, and since both Docker
>> and the Ubuntu machine has the same view of the files, everything works
>> seamlessly.
>>
>> When working from home, acme is not an ideal solution, since the VPN
>> network has high latency and relatively low bandwith (a fraction of my
>> 500Mbit fiber connection). I run an X server on Windows, and start acme on
>> the Ubuntu machine. Editing files works fairly well, but commands which
>> spew large amounts of text really bogs acme down. I could switch to VSCode,
>> but I would prefer not to.
>>
>> Now, I have experimented with other solutions which could give me a
>> better experience. Running sshfs and a local acme would be one possibility,
>> but it is not really fast enough for searching through the code. (I can
>> search the 5M lines code base in about 2 seconds with

Re: [9fans] venti/mirrorarenas usage

[Ole-Hjalmar Kristensen]

I just stumbled across this thread, so I will repost something I wrote in
2017 about mirroring of venti. I had the need to create the union of two
venti servers, so I wrote a client that copies individual venti blocks,
given the scores. This you can do without worrying about what is on the
target venti, since the deduplication takes care of any conflicts. So, in
case it might be useful for someone, here it comes:

Based on copy.c and readlist.c, I have cobbled together a venti client to
copy a list of venti blocks from one venti server to another. I am thinking
of using it to incrementally replicate the contents on one site site to
another. It could even be used for two-way replication, since the CAS and
deduplicating properties of venti ensures that you will never have write
conflicts at a block level.

I have tried it out by feeding it with the output from printarenas, and it
seems to work reasonably well. Does anyone have any good ideas about how to
incrementally extract the set of scores that has been added to a venti
server? You could extract the whole set of scores and do a diff with an old
set of course, but that's rather inefficient.

Ole-Hj.


#include 
#include 
#include 
#include 
#include 

enum
{
// XXX What to do here?
VtMaxLumpSize = 65535,
};

char *srchost;
char *dsthost;
Biobuf b;
VtConn *zsrc;
VtConn *zdst;
uchar *buf;
void run(Biobuf*);
int nn;

void
usage(void)
{
fprint(2, "usage: copylist srchost dsthost list\n");
threadexitsall("usage");
}

int
parsescore(uchar *score, char *buf, int n)
{
int i, c;

memset(score, 0, VtScoreSize);

if(n != VtScoreSize*2){
werrstr("score wrong length %d", n);
return -1;
}
for(i=0; i= '0' && buf[i] <= '9')
c = buf[i] - '0';
else if(buf[i] >= 'a' && buf[i] <= 'f')
c = buf[i] - 'a' + 10;
else if(buf[i] >= 'A' && buf[i] <= 'F')
c = buf[i] - 'A' + 10;
else {
c = buf[i];
werrstr("bad score char %d '%c'", c, c);
return -1;
}

if((i & 1) == 0)
c <<= 4;

score[i>>1] |= c;
}
return 0;
}

void
threadmain(int argc, char *argv[])
{
int fd, i;

ARGBEGIN{
default:
usage();
break;
}ARGEND

if(argc < 2)
usage();

fmtinstall('V', vtscorefmt);
buf = vtmallocz(VtMaxLumpSize);

srchost = argv[0];
zsrc = vtdial(srchost);
if(zsrc == nil)
sysfatal("could not dial src server: %r");
if(vtconnect(zsrc) < 0)
sysfatal("vtconnect src: %r");

dsthost = argv[1];
zdst = vtdial(dsthost);
if(zdst == nil)
sysfatal("could not dial dst server: %r");
if(vtconnect(zdst) < 0)
sysfatal("vtconnect dst: %r");

if(argc == 2){
Binit(&b, 0, OREAD);
run(&b);
}else{
for(i=2; i wrote:

> Thank you very much guys, there are some helpful answers,
> even though this thread got a little messy. (didn't see that coming)
> 
> Charles Forsyth, since you are using venti/mirrorarenas,
> if you don't mind, I would like to know what your venti.conf looks like.
> 
> -marco
> 

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/Tca0eb0fbb2404e31-Ma9856351c69b05b0716dccc0
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription