This blog entry documents my recent (successful) attempt to use Simon Urbanek's Rserve and FastRWeb for CGI scripting with R. This is a working blog entry and will be updated or replaced as needed (last updated 4:15 PM 10/6/2011).
#### Helpful documentation:
http://rforge.net/FastRWeb/
http://urbanek.info/research/pub/urbanek-iasc08.pdf
http://www.rforge.net/Rserve/
http://cran.r-project.org/web/packages/Rserve/
(Plus personal communications with Simon, the results
of which are included in the summary below)
#### The steps used (your configuration probably varies):
0. Ubuntu Linux, 64-bit, Version 10.04 LTS (plus updates). I did the following steps as root, but will return to security issues below.
1. I did a fresh installation of the apache2 web server. I noted that the default location of the cgi-bin (used later) is /usr/lib/cgi-bin; yours may vary. I confirmed that this was up and running and that I could use the toy CGI script foo.cgi placed in the cgi-bin:
#!/usr/bin/perl
print "Content-type: text/html\n\n";
print "<html>Hello World</html>";
To test this I pointed my browser to http://localhost/cgi-bin/foo.cgi; if there are problems, consult your system administrator or do detective work (probably in the log files, /var/log/apache2 on my system). Do not continue until you have Hello World working!
2. I did a fresh installation of R, version 2.13.2, using the required --enable-R-shlib option to configure.
3. I installed R packages Rserve, Cairo, FastRWeb, and (though not required) XML (this required installing some libxml2... package in Ubuntu, first, but again is NOT required for Rserve/FastRWeb).
4. After installing FastRWeb, I went into the inst directory of the package and ran the install.sh script; this created /var/FastRWeb, used extensively below.
5. I went into /var/FastRWeb/code and examined the files; in a slightly older version of FastRWeb I commented out a few lines, but the current (10/6/2011) version removed that need for me.
6. I fired up R, and per Simon's instructions did the following:
system.file("cgi-bin", package="FastRWeb")
This revealed the location of a binary called Rcgi. I copied this into /usr/lib/cgi-bin, and renamed it R (instead of Rcgi).
7. Finally, I created a file /var/FastRWeb/web.R/foo.png.R:
# foo.png.R:
run <- function(n=100, ...) {
n <- as.integer(n)
p <- WebPlot(800, 600)
plot(rnorm(n), rnorm(n), pch=19, col=2)
p
}
8. I tested it with the URL: http://localhost/cgi-bin/R/foo.png?n=500
#### Security Issues
I have a feeling that if you have a "trusted machine" without user access, the steps above may not technically pose security risks (even as root); but they do not represent good security practices and *would* introduce security risks on shared servers. For my purposes, I added to the beginning of /var/FastRWeb/code/rserve.conf:
gid 33
uid 33
because www-data (uid and gid 33) is the username for my apache2 instances and it seemed like a reasonable choice. For good measure, I also changed permissions in /var/FastRWeb:
chown www-data:www-data .
chown -R www-data:www-data ./*
Finally, I set
sockmod 0660
umask 0007
based on Simon's recommendation for further security. To stop Rserve and FastRWeb:
killall -INT Rserve
Thank you for these instructions. I am trying to install it now on my web server.
ReplyDeleteNow that you have used it for a couple of months, what do you think about the performance using CGI? Does it work well for you?
C Butler
I'm very impressed. I'm close to going "live" with a new web site for a major project at Yale to be announced at the World Economic Forum in Davos later in January, and the data explorer is built with FastRWeb.
ReplyDeleteI've learned quite a few tricks, and will propose a paper with Simon that might help folks in the end. Feel free to email me directly with a question (john.emerson@yale.edu) and I'll do my best. I will try to remember to link the new site to this blog. As you can see, I'm not a very active blogger.
Jay
How is the new website for the Yale project going? Is it live? Got a link?
DeleteAnd the paper with Simon?
I followed your instructions and was able to get Step 1 working successfully. However, my webhost provider does not give me root access. So I couldn't install R in step 2.
I'm looking for a solution to this -- any recommendations?
See http://epi.yale.edu. Graphics within the Country Profiles are produced on-demand using grid graphics (as is all the HTML with the iframes).
DeleteAs for the permissions problems, I'm not sure -- it could be that if Apache is set up with certain permissions, and you are given appropriate permissions, this might be possible without root access.
Hi Dude,
ReplyDeleteGood Post! For some strange reason some users are still using the old, unsupported JRclient which has been replaced three years ago. Thanks for sharing it.......
Website Spider Software
Hello,
ReplyDeleteI adapted and translated your post in French on my new blog.
http://www.ronanm.name/fr/2012/02/fastrweb-mac/
Thank you for this post.
I am trying to integrate R in a website and am in the process of exploring the possibilities. I read about FastRWeb and found it very interesting. Since I am also using Ubuntu I was led to your blog post by my search
ReplyDeleteHowever, I am using Xampp as my developmental server, I think I would need some different steps to get it working.
I installed FastRWeb along with the required packages and ran install.sh as instructed. I then located Rcgi and copied it to /opt/lampp/cgi-bin and renamed it R. /opt/lampp/cgi-bin the default location of Xampp's cgi-bin. I have tested that it is up and running by running the test cgi script in Xampp by entering the url "localhost/cgi-bin/test-cgi"
I created foo.png.R as in your example and placed it in /var/FastRWeb/web.R/ (I have chmod 777 on this folder as it is a local server and only I am using my computer so security is not a big concern)
When tested with "http://localhost/cgi-bin/R/foo.png?n=500" I got the error "Unable to connect (result=-1, socket:unknown socket error 2). "
I tried setting "gid 33 uid 33 sockmod 0660 umask 0007" in /var/FastRWeb/code/rserve.conf without differenc. I suppose these parameters would be different for my setup but I don't know how to determine these parameters for my system.
Your assistance will be greatly appreciated. Thanks.
I have the same error message. Did you make any progress here?
DeleteI think this is a problem with the ownership/group/permissions of the directories in /var/FastRWeb. You might also want to stop FastRWeb and manually delete /var/FastRWeb/socket before restarting.
DeleteHere's my setup on my Ubuntu development box:
jay@bayesman:/var/FastRWeb$ ls -als
total 48
4 drwxr-xr-x 7 www-data www-data 4096 2012-04-16 14:02 .
4 drwxr-xr-x 17 root root 4096 2011-10-02 13:05 ..
4 drwxrwxr-x 2 www-data www-data 4096 2012-04-15 13:44 code
4 drwxrwxr-x 2 www-data www-data 4096 2012-01-11 08:53 misc
0 srw-rw---- 1 www-data www-data 0 2012-04-16 14:02 socket
24 drwxrwxr-x 2 www-data www-data 20480 2012-04-16 14:02 tmp
4 drwxrwxr-x 2 www-data www-data 4096 2011-11-07 15:45 web
4 drwxrwxrwx 3 www-data www-data 4096 2012-03-24 17:08 web.R
Thanks Jay for your answer and for providing these detailed instructions in the first place! I finally found the solution: On an ubuntu installation your description works perfectly fine, but I was working on a CentOS copy where SElinux seems a bit stricter by default and prevents CGI scripts from opening network connections, effectively blocking the R CGI client from reaching Rserve. Here is how to change the SElinux config:
Deletehttp://bioinfogeek.blogspot.com/2012/04/fastrweb-on-centos.html
I'm taking part of this back. This helps you if you want to establish a TCP/IP connection. With unix sockets it's not helping. I have changed the user properties of the files according to yours with user apache instead of www-data on CentOS:
Delete4 drwxr-xr-x. 6 apache apache 4096 Apr 20 15:24 .
4 drwxr-xr-x. 19 root root 4096 Apr 20 10:57 ..
4 drwxr-xr-x. 3 apache apache 4096 Apr 20 15:24 code
0 srw-rw-rw-. 1 apache apache 0 Apr 20 15:24 socket
4 drwxr-xr-x. 3 apache apache 4096 Apr 20 13:07 tmp
4 drwxr-xr-x. 3 apache apache 4096 Apr 19 12:05 web
4 drwxr-xr-x. 3 apache apache 4096 Apr 20 15:08 web.R
Unfortunately no progress, but I wanted to use TCP/IP anyway.
1. I had the same environment as yours Ubuntu and LAMPP. I got the asme error Unable to connect (result=-1, socket:unknown socket error 2).
Delete2. The as pointed By Jay, I tried to start FastRweb
/var/FastRWeb/code/start
It gave me error
R CMD Rserve --RS-conf /var/FastRWeb/code/rserve.conf --vanilla --no-save/usr/lib/R/bin/Rcmd: 62: exec: Rserve: not found
helpful link : http://svn.rforge.net/FastRWeb/trunk/INSTALL
3. The I copied the file to the directory
sudo cp /home/amitabh/R/x86_64-pc-linux-gnu-library/3.0/Rserve/libs/Rserve /usr/lib/R/bin/Rserve
4. The when I started FastRweb
/var/FastRWeb/code/start
it started successfully, and I was able to see Hello World in http://localhost/cgi-bin/R/main
Hope this helped. Thank you.
Thanks for this. I keep getting this error
ReplyDeleteError in strsplit(request$raw.cookies, ";", fixed = TRUE) :
object 'request' not found
(from the command line) and
Evaluation failed with error code -10
shows up in the browser when executing the foo.png.R script (or anything for that matter). Any idea what might be the cause?
Is this a Linux installation? Were all the previous steps followed in the order described? I have to say, I have never seen this before. When you say, "from the command line" what do you mean? Aren't you trying to visit the page in your browser, like
ReplyDeletehttp://localhost/cgi-bin/R/foo.png?n=500
?
I got two seperate error messages, one through the browser (Evaluation code with error code -10) and the other other appeared on the command line where Rserve had been invoked.
ReplyDeleteAnyway ,I removed the common.R script which was part of the installation, and I managed to get my own scripts working. Seems that it has something to do with the request object actually not being available.
Thanks for this great blog post though!
Hi Jay:
ReplyDeleteI keep getting the Error message
Unable to connect (result=-1, socket:unknown socket error 2).
Does this sound familiar, please?
Thanks,
Erin
Perhaps you didn't things up by running the script ..../FastRWeb/code/start ... and if you did, make sure there are no error messages.
ReplyDeleteIf you did do that (and if there are no error messages), then I wonder if you have a "stale" socket left over from a previous run? Make sure you do the killall as documented in my blog, and then check to see if there is an object .../FastRWeb/socket. If there is, delete it. Then start up again and try it.
This comment has been removed by the author.
DeleteHey Jay, hey everybody,
DeleteI have got a similar problem as Erin had.
My system: ubuntu 12.04 server, apache 2.2 webserver and R 3.0.1.
If I run /var/FastRWeb/code/start in the terminal I get:
Starting Rserve on xxx
Loading packages...
XML: TRUE
Cairo: TRUE
Matrix: TRUE
FastRWeb: TRUE
Rserv started in daemon mode.
##> SOCK_ERROR: bind error #13(access denied)
and I have no socket-file in /var/FastRWeb.
When I run the start-script /var/FastRWeb/code/start as root,
everything seems to be fine.
But I ask myself if it is usual / legal / safe to run the start-script as root?!
Especially because on http://rforge.net/Rserve/doc.html it is said that "never ever run Rserve as root...".
What do you think?
Thank you very much for your help!
Greets!
Harry
I'm sorry, I have no idea. I've move to a Mac, myself, and am just doing development instead of a production environment -- so I use root. The only advice I have is to check the owner/permissions on every file and directory that you are working with (including apache).
DeleteJay
Hi again!
ReplyDeleteFinally figured out how to start the Rserve
However (isn't there always a however) now I'm getting an error
##> SOCK_ERROR: bind error #13(access denied)
Whoops...ran start as root, all is well.
Sorry for the trouble.
Hi Jay:
ReplyDeleteHave you ever tried this with a Macbook air, please?
When I try to get the http://localhost (with foo.png), I get a 'file not found on this server."
Have you run into this please?
Thanks,
Erin
I'm sorry, there isn't enough information. I don't use a Mac, both others have used it successfully. It isn't clear whether (a) your web server is running -- I think it probably is, (b) you started FastRWeb -- unclear, just remember you probably have to restart every time you reboot your Mac, and (c) whether you gave the right path. It needs to be specified correctly, likely including cgi-bin/R/ as part of the path.
ReplyDeleteHi Jay!
ReplyDeleteIt was my fault...dealing with the Rcgi.
But it works on a Mac both on Firefox and Safari, which is good 2 know.
Thanks,
Erin
Hi again Jay!
ReplyDeleteI'm running this on a "real" server (not localhost) and I keep getting "Error in file(filename, "r", encoding = encoding) : cannot open the connection" as my message. I assume that it can't find foo.png.R.
I have sourced in into several directories, but to no avail. Have you run across this, please?
Thanks,
Erin
Never mind....I was using foo.png.R and not foo.png.
ReplyDeleteWhat a doofus.
Thanks,
Erin
Jay,
ReplyDeleteWhen trying to run the foo.png.R script, I get the following error:
Evaluation failed with error code -10
I am having a hard time finding a solution. Have you heard or seen this error? If so can you tell me what would resolve it?
Thanks.
I did a quick Google search and it may be related to 32-bit versus 64-bit installations of FastRWeb? Make sure you started FastRWeb and that this startup was successful without errors.
ReplyDeleteJay
Yes. FastRWeb is started. The issue I see after starting the package is:
DeleteStarting Rserve on (myserver)
Loading packages....
XML: FALSE
Cairo: TRUE
Attaching package: 'Matrix'
The following object(s) are masked from 'package:base':
det
Matrix: TRUE
FastRWeb: TRUE
Rserv started in daemon mode.
Can you do a simple "hello world" example, or the ones provided by FastRWeb, just to establish that sometime works? It isn't clear if there is a specific problem with your example, or a general problem.
DeleteYes. Everything is working up to the last test which is the Foo script. I get the same error testing the default scripts as well, i.e., main.R
DeleteFollowing your instructions and using a new Ubuntu 12.04 LTS Server (32-bit ISO downloaded from Ubuntu), I was able to install FastRWeb as follows:
ReplyDeletesudo apt-get update && sudo apt-get upgrade
sudo apt-get install openssh-server
sudo apt-get install libapache2-mod-php5
sudo apt-get install r-base
sudo apt-get install r-cran-rserve
sudo apt-get install libcairo2-dev
sudo apt-get install libxt-dev
sudo R
install.packages("FastRWeb",dep=TRUE)
q()
cd /usr/local/lib/R/site-library/FastRWeb
sudo ./install.sh
sudo /var/FastRWeb/code/start
sudo cp /usr/local/lib/R/site-library/FastRWeb/cgi-bin/Rcgi /usr/lib/cgi-bin/R
Go to http://{server IP}/cgi-bin/R/example1.png .
The following fix for broken symlinks was necessary after a server reboot before FastRWeb could be started:
Deletesudo rm /usr/lib/R/bin/Rserve
sudo ln -s /usr/lib/R/site-library/Rserve/libs/Rserve /usr/lib/R/bin/Rserve
sudo rm /usr/lib/R/bin/Rserve.dbg
sudo ln -s /usr/lib/R/site-library/Rserve/libs/Rserve.dbg /usr/lib/R/bin/Rserve.dbg
sudo /var/FastRWeb/code/start
This comment has been removed by the author.
ReplyDeletehello
ReplyDelete