GUI API & Perl

Message boards : API : GUI API & Perl
Message board moderation

To post messages, you must log in.

AuthorMessage
River~~
Avatar

Send message
Joined: 12 Mar 07
Posts: 59
Message 8832 - Posted: 17 Mar 2007, 14:33:16 UTC

I am writing some perl scripts at present, stuff like waiting for a box to checkpoint and only then shutting down BOINC and doing a backup. Another example is doing a simple display of the progress of the results so far.

The scripts access info from local and remote machines so the RPC makes more sense than a direct read of the .xml

I am wondering if anyone has any advice on talking to the client with the RPC from perl? If not (or if the advice is "don't"!) then I will stick with the prototype which is to use boinc(_)cmd to do the talking. Like where do I look for the RPC calls, etc

The downside of this is means an extra overhead in each RPC, to fire up a new shell every few seconds in which to run boinc(_)cmd, but made it easy to code for the prototype.

River~~
ID: 8832 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1174
Argentina
Message 8849 - Posted: 17 Mar 2007, 23:02:34 UTC

If you know how to open an outgoing TCP port and write XML to it from Perl, you're on luck! :) The XML protocol isn't documented, but running Wireshark with BOINC Manager running, and giving a quick look at gui_rpc_client_ops.C, may give useful information. As with all other parts of BOINC, the stupid XML parser requires each tag on a separate line (I have heard it's not really the case with GUI RPCs, because of the way the data is read - but don't take my word for it).
ID: 8849 · Report as offensive
River~~
Avatar

Send message
Joined: 12 Mar 07
Posts: 59
Message 8851 - Posted: 17 Mar 2007, 23:57:04 UTC - in response to Message 8849.  
Last modified: 17 Mar 2007, 23:57:47 UTC

If you know how to open an outgoing TCP port


IO::socket - it opens a filehandle FH on the given port number, then you print FH and read FH just as if it were a file.

My difficulty is knowing exactly what to say, in what order.

and write XML to it from Perl


by hand, with print, yes; no doubt there is a perl module for XML but I don't know my way around any

...Wireshark...


useful tip, thanks;

...giving a quick look at gui_rpc_client_ops.C...


have you got a URL for that please? - I found the .h files on the website in the lib directory, but couldn't find either the relevant C sources to go with the headers. I'd also like to look at the C source for boinc_cmd if you can point me in the right direction...

River~~


ID: 8851 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1174
Argentina
Message 8852 - Posted: 18 Mar 2007, 2:02:57 UTC

They are on the lib directory too. boinc/lib/boinc_cmd.C, boinc/lib/gui_rpc_client_ops.C, and there are others in boinc/lib that may be interesting. There is also boinc/client/gui_rpc_server_ops.C, which is the core-client-side of the story :) That's the part that parses manager requests and gives the replies.

Something I forgot to say, but it's important: as noted on the bottom of the (brief) API documentation for GUI RPCs, requests should be finished with \003 (that is, a single byte of value 0x03). I have noticed in practice that the core client doesn't really care about that finishing byte, but it's always better to follow protocol specs correctly, in case they make it really required.
ID: 8852 · Report as offensive
River~~
Avatar

Send message
Joined: 12 Mar 07
Posts: 59
Message 8968 - Posted: 21 Mar 2007, 19:18:58 UTC - in response to Message 8852.  

... requests should be finished with ... a single byte of value 0x03. I have noticed in practice that the core client doesn't really care about that finishing byte

at least when the preceeding items are correctly presented.

When the preceeding items are wrong for some reason, like River making up the coding as it goes along, the ^C / chr(3) / 0x03 stops the client falling off the end, which is another reason why, as you say
... it's always better to follow protocol specs correctly ...


;-)

Thanks for the pointers to the C code - the most helpful thing to find was the function that prepares the hash that is used in the <auth2> response - there was no way to work that out from analysis of passing packets. I'd noticed it was a hex number of the right length to be an MD5, and hopefull I'd have guessed to simply concat the challenge with the password, but it makes it a lot easier to read the C and *know* what effect the code needs to have.

River~~
ID: 8968 · Report as offensive
River~~
Avatar

Send message
Joined: 12 Mar 07
Posts: 59
Message 8972 - Posted: 21 Mar 2007, 22:29:59 UTC

OK, here is a demo program to help others up the learning curve I just climbed in this thread. Big thanks again to Nicolas. Enjoy!

#!/usr/bin/perl -w
#
# boincdemo
#
# (c) River 2007
#
# GPL applies, please see http://www.gnu.org/licenses/gpl.txt
#
# This script connects to the chosen BOINC client, authorizes,
# and then allows the user to enter commands as required

use IO::Socket;
use Digest::MD5 qw(md5_hex);

print "host? (name or IP or <return> for localhost)\n";

$host=<STDIN>;
if ( $host =~ /^\n$/ ) {
$host = 'localhost';
} else {
$host =~ s/\n//;
}

my $client = new IO::Socket::INET (
PeerAddr => $host,
PeerPort => '31416',
Proto => 'tcp' );

die "could not open socket to host $host" unless $client;

print "pwd? (<return> to use pwd from file in this directory)\n";

$pwd=<STDIN>;

if ( $pwd =~ /^\n$/ ) {
open (PWD, '< ./gui_rpc_auth.cfg') || die "can't read password file $!";
$pwd = <PWD>;
close PWD;
}
$pwd =~ s/\n//;

print $client "<boinc_gui_rpc_request>\n",
"<auth1/>\n",
"</boinc_gui_rpc_request>\n\003";

{
local $/ = "\003";
$reply = <$client>;
}

$reply =~ ?<nonce>(.*)</nonce>?;
$nonce = $1;
$hash = md5_hex($nonce, $pwd);
print $client "<boinc_gui_rpc_request>\n",
"<auth2>\n",
"<nonce_hash>$hash</nonce_hash>\n",
"</auth2>\n",
"</boinc_gui_rpc_request>\n\003";

{
local $/ = "\003";
$reply = <$client>;
}
die "password rejected by $host" unless $reply =~ ?<authorized/>?;

print "\nConnected to $host\n\n",
"Please enter XML for client\n",
"blank line to receive reply\n",
"(gui_rpc tags added automatically)\n\n";

$cmd = <STDIN>;

do {
print $client "<boinc_gui_rpc_request>\n",
$cmd;
until ( $cmd =~ /^$/ ) {
print $client $cmd;
$cmd = <STDIN>;
}
print $client "</boinc_gui_rpc_request>\n\003";

{
local $/ = "\003";
$reply = <$client>;
}
print $reply;

print "\nPlease enter more XML, or blank line to exit\n\n";

$cmd = <STDIN>;
} until ( $cmd =~ /^$/ );

close $client;
1
ID: 8972 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1174
Argentina
Message 8973 - Posted: 21 Mar 2007, 22:39:45 UTC
Last modified: 21 Mar 2007, 22:42:19 UTC

Just because I can:

#!/usr/bin/perl -w
#
# boincdemo
#
# (c) River 2007
#
# GPL applies, please see http://www.gnu.org/licenses/gpl.txt
#
# This script connects to the chosen BOINC client, authorizes,
# and then allows the user to enter commands as required

use IO::Socket;
use Digest::MD5 qw(md5_hex);


print "host? (name or IP or <return> for localhost)n";

$host=<STDIN>;
if ( $host =~ /^n$/ ) {
$host = 'localhost';
} else {
$host =~ s/n//;
}


my $client = new IO::Socket::INET (
PeerAddr => $host,
PeerPort => '31416',
Proto => 'tcp' );

die "could not open socket to host $host" unless $client;


print "pwd? (<return> to use pwd from file in this directory)n";

$pwd=<STDIN>;

if ( $pwd =~ /^n$/ ) {
open (PWD, '< ./gui_rpc_auth.cfg') || die "can't read password file $!";
$pwd = <PWD>;

close PWD;
}
$pwd =~ s/n//;

print $client "<boinc_gui_rpc_request>n",
"<auth1/>n",
"</boinc_gui_rpc_request>n\003";


{
local $/ = "\003";
$reply = <$client>;
}

$reply =~ ?<nonce>(.*)</nonce>?;
$nonce = $1;
$hash = md5_hex($nonce, $pwd);
print $client "<boinc_gui_rpc_request>n",

"<auth2>n",
"<nonce_hash>$hash</nonce_hash>n",
"</auth2>n",
"</boinc_gui_rpc_request>n\003";

{
local $/ = "\003";
$reply = <$client>;
}

die "password rejected by $host" unless $reply =~ ?<authorized/>?;

print "nConnected to $hostnn",
"Please enter XML for clientn",
"blank line to receive replyn",

"(gui_rpc tags added automatically)nn";

$cmd = <STDIN>;

do {
print $client "<boinc_gui_rpc_request>n",
$cmd;
until ( $cmd =~ /^$/ ) {

print $client $cmd;
$cmd = <STDIN>;
}
print $client "</boinc_gui_rpc_request>n\003";

{
local $/ = "\003";
$reply = <$client>;
}

print $reply;

print "nPlease enter more XML, or blank line to exitnn";

$cmd = <STDIN>;
} until ( $cmd =~ /^$/ );

close $client;
1
ID: 8973 · Report as offensive
River~~
Avatar

Send message
Joined: 12 Mar 07
Posts: 59
Message 8982 - Posted: 22 Mar 2007, 7:13:44 UTC - in response to Message 8973.  
Last modified: 22 Mar 2007, 7:20:40 UTC

Just because I can:...


:-)
R~~

It makes my code look a lot better, thank you!

edit - PS
Anyone copying the code please copy from the monocolour posting, as the backslashes on all the newlines got eaten in the copying / colourisation.
What is really odd is that the backslashes on the ctrl-c's ( \003 )
survived.
ID: 8982 · Report as offensive
River~~
Avatar

Send message
Joined: 12 Mar 07
Posts: 59
Message 8984 - Posted: 22 Mar 2007, 9:17:23 UTC

By the way, the script is not intended to be production quality but simply a demo of how to set up a channel that can be read and written to, and one way to code the auth steps in perl.

I have already noticed two more bugs, and there are bound to be more, which I will leave the reader to find. The two I now know about are:

Sometimes the client gets upset by the message arriving in chunks, and it would therefore be better to concat all the lines into a single variable and print in one go.

It does not test the sends to see if they work, you might want to do this, one way would be something like

(print $client $message) || die ".... $!";

R~~
ID: 8984 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1174
Argentina
Message 8990 - Posted: 22 Mar 2007, 13:55:40 UTC - in response to Message 8982.  


Anyone copying the code please copy from the monocolour posting, as the backslashes on all the newlines got eaten in the copying / colourisation.
What is really odd is that the backslashes on the ctrl-c's ( \003 )
survived.

I noticed the \003 were showing as [weird character]03 on the textbox when I clicked Edit, so I fixed them manually. Didn't notice anything about \n's though. But it's quite weird. Try replying to my post and see what shows up. This is what I got when I replied your post.
ID: 8990 · Report as offensive
River~~
Avatar

Send message
Joined: 12 Mar 07
Posts: 59
Message 8998 - Posted: 22 Mar 2007, 22:15:49 UTC - in response to Message 8990.  

... Try replying to my post and see what shows up. ...


In Firefox on win2k they go to "?03"

In Firefox on KDE (Debian Linux) they show more than one behaviour
- in the setting of $/ they go to "03" (odd char disappears totally)
- in the print strings they go to a different weird symbol to yours

In IE6 on win2k they go to "03" (odd char disappears totally)

In Konqueror (KDE, Debian Linux) they go to " 03" (odd char becomes space)

The strange character must be produced by some layer in the CGI mistakenly applying the backslash and then being re-transmogrified by the various browsers.

I wonder if this is connected with two other oddities in the forum software
- the way apostrophes sometimes pick up backslashes for no apparent reason
- the way extra lines are interpolated into [ code ] and [ pre ] blocks

R~~
ID: 8998 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1174
Argentina
Message 9232 - Posted: 30 Mar 2007, 17:43:36 UTC
Last modified: 30 Mar 2007, 17:45:01 UTC

I had an idea to make a wrapper program for XML over HTTP. Wrapper opens port 12345 (I'll use the number on my explanation, most probably I won't really use that one) and listens on it. It should behave like a webserver. Another program using it would send an XML request (like the ones the GUI RPC uses) via POST to http://localhost:12345/boinc or something similar. My wrapper would connect to the BOINC RPC port and pass the request over, get the client's reply, and return it as content via HTTP.

Also, it would set a sessionID cookie for the authentication. That way I can have multiple programs talking to my wrapper, and the wrapper would have multiple connections to the client; using the cookie it can tell which connection to send the request to.

Now here's the problem: If a program sends my wrapper two requests almost at the same time, naturally using the same cookie, what would the wrapper do? Only thing I can do is send one, get the reply from client, forward it back, and only then send the second one. In the stupid way the client code for RPCs is made, I can't do "pipelining" like on HTTP. To prove it's Doing It Wrong: most webservers support HTTP pipelining just well, even before the whole idea of using pipelines appeared. That's just by following socket programming rules and good practices correctly.

I should go complain some more to the devs. I'm not very optimist on this being modified to work correctly.

PS: If you're curious on what I need such wrapper for, stay curious. You'll know when I want you to know. It's more fun that way :)
ID: 9232 · Report as offensive

Message boards : API : GUI API & Perl

Copyright © 2018 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.