Hi,
I see there has been some discussion in this list about cheating remote.
In computerchess this toleration has grown out of hand.
Setting the rules clear and sharp there in computer-go might avoid
for the future a lot of problems.
There is a very simple manner to avoid cheating in go.
But let me adress a few points first.
1) neural nets
forget about neural nets and cheating. A year or 12+ ago we had a lot
of neural net guys in computerchess
as well. ANN's are not even close to representing the human mind, as
modern insights in how brains work shows
clearly already for quite a while. Most important is that the
automatic learning techniques of neural nets are so
total inefficient that it is really difficult to use them well.
Soon anything that is neural net will be beaten by the rest major
league.
The only case i remember that was a tad more stubborn was basically
someone who tried to fool the rest;
he bought source code from someone and sold that as the 'neural net
optimized version' engine. Yet the original
programmer of that code (Joost Buijs), he assured me that this
program definitely used his parameters and not
some neural net optimized parameters, as he could reproduce even
every score of it.
So in that case it was not the neural net that was there, it just was
getting used as a sales reason.
Yet the neural nets will get beaten major league. If not next year,
then some years later.
You can't forever improve a product without good debugging methods of
what it actually is doing.
A black box that is real clever and intelligent doesn't exist.
2) sure on paper it is really easy to cheat.
IT IS ALSO REALLY EASY TO CHEAT WITHOUT REMOTE MACHINES IN FACT.
Oh in computerchess we've seen it all. There is a certain species of
persons on the planet, they are not in big numbers there,
who are capable of fooling in a professional manner other persons,
James Bond is nothing compared to the sneaky manners they
get things done.
For sure a bunch of them will also try it in computer-go.
For these guys, considering how weak for the coming few years go
computers will play, there is not a big difference between remote
machines and local machines. It's too easy to cheat for them.
Communication to and from a program is too difficult to 100% monitor.
So to speak just keeping the mouse at a certain spot is already
enough to cheat, or having someone a fewmeters away at his laptop
signal something over blue tooth or whatever to the playing machine.
All been done.
The only real simple manner of catching crooks is by having a good
tournament director who will enforce in case of suspected
moves played, that an engine must reproduce the move it played, with
some reasonable decent score.
Now some of you will argue loud in one choir: "parallel search
doesn't reproduce moves".
One move can make or break a game, yet those who cheat have the habit
to cheat many moves a game
and for several games. So there should be many datapoints one
complains about.
If a program basically cannot reproduce a move, at the discretion of
the tournament leader who might want to see whether
a move in question has a very similar score to other alternatives (in
which case of course you don't know which of the equal
scored moves or nearly equal scored moves can get played).
But the principle thing is reproduction of great moves.
If a program under no circumstance can reproduce a specific move and
that for several occasions, then that's very clear proof of course.
That is a rule that should be introduced in computerchess also IMHO.
Note there is also the time constraint. And search depth constraint.
One always must have a logfile displaying which iterations or steps a
program already has performed; if one searches in a
very selective manner, a rather easy form of cheating that is 'near
undetectable', is when a program plays moves that it
normally spoken would not have found on that hardware, yet iterations
deeper.
As in selective search you can assume that in a move sequence m0..mN
that the moves 0..N represent the line that one needs
to see to find it, there will be of course selectively moves in that
sequence that might have been reduced somehow.
So if one takes care that in the 'hashtable' such sequence already
gets searched deeper, by manually enforcing that sequence,
then the program 'learns' itself from hashtable sooner that move.
Now in chess this is easier than go currently, as the search method
used is reductions,
but it'll come in go also.
Really effective is giving in the 'mainlines' of your opponent to be
searched fully by a number of cores.
Yet again the only way to really detect this is by forcing
reproduction of the moves by the tournament director.
If a system can't reproduce enough of the great moves played, for
whatever reason, bad luck.
For parallel systems that search total non-deterministic, there is
also a simple lemma; the distribution of good/bad moves
should still be in proportion with the game.
Explanation: if a program plays 30 bizarre moves and 10 suspected
strong moves a game, then of course when
retrying the positions onto the program by the tournament director in
case of an investigation, it again should produce 30 bizarre moves
and 10 suspected strong moves. It cannot be the case that it produces
40 bizarre moves suddenly or 50 bizarre moves and 0 strong
moves, as that would add to the suspicion.
Statistics prove everything here.
For the coming years there is plenty of really strong go players
around to judge quality.
The big advantage of go over chess here is the huge number of moves
that get played in 1 game.
Thanks,
Vincent
On Feb 2, 2009, at 11:25 AM, Nick Wedd wrote:
In message
<262b2f900902010529r2ddec4afq31705bd9ccfda...@mail.gmail.com>, Erik
van der Werf <erikvanderw...@gmail.com> writes
< snip >
Something else for the discussion. I would like to have a rule about
mandatory displaying the thinking process of the program so that both
operators have an idea of what is happening. Especially for remote
play I think this is needed because now it is just too trivial to
cheat.
I understand the need to try to avoid cheating. But I am sceptical
about the effectiveness of this method.
1.) A neural net cannot explain its "thinking process" because it
does not have any.
2.) It would still be too easy to cheat. The cheater could run a
program which looks at the position and generates a plausible
"display of its thinking process", while a professional player
thinks and then tells it where to play. Then the program generates
more "display of thinking process" tending to support the
recommended move, before playing it.
<snip>
Nick
--
Nick Wedd n...@maproom.co.uk
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
