Thursday, October 6, 2011

Setting up FastRWeb/Rserve on Ubuntu

This blog entry documents my recent (successful) attempt to use Simon Urbanek's Rserve and FastRWeb for CGI scripting with R.  This is a working blog entry and will be updated or replaced as needed (last updated 4:15 PM 10/6/2011).

#### Helpful documentation:

    http://rforge.net/FastRWeb/
    http://urbanek.info/research/pub/urbanek-iasc08.pdf
    http://www.rforge.net/Rserve/
    http://cran.r-project.org/web/packages/Rserve/
    (Plus personal communications with Simon, the results
     of which are included in the summary below)

#### The steps used (your configuration probably varies):

0. Ubuntu Linux, 64-bit, Version 10.04 LTS (plus updates).  I did the following steps as root, but will return to security issues below.

1. I did a fresh installation of the apache2 web server.  I noted that the default location of the cgi-bin (used later) is /usr/lib/cgi-bin; yours may vary.  I confirmed that this was up and running and that I could use the toy CGI script foo.cgi placed in the cgi-bin:

    #!/usr/bin/perl
    print "Content-type: text/html\n\n";
    print "<html>Hello World</html>";

To test this I pointed my browser to http://localhost/cgi-bin/foo.cgi; if there are problems, consult your system administrator or do detective work (probably in the log files, /var/log/apache2 on my system).  Do not continue until you have Hello World working!

2. I did a fresh installation of R, version 2.13.2, using the required --enable-R-shlib option to configure.

3. I installed R packages Rserve, Cairo, FastRWeb, and (though not required) XML (this required installing some libxml2... package in Ubuntu, first, but again is NOT required for Rserve/FastRWeb).

4. After installing FastRWeb, I went into the inst directory of the package and ran the install.sh script; this created /var/FastRWeb, used extensively below.

5. I went into /var/FastRWeb/code and examined the files; in a slightly older version of FastRWeb I commented out a few lines, but the current (10/6/2011) version removed that need for me.

6. I fired up R, and per Simon's instructions did the following:

    system.file("cgi-bin", package="FastRWeb")

This revealed the location of a binary called Rcgi.  I copied this into /usr/lib/cgi-bin, and renamed it R (instead of Rcgi).

7. Finally, I created a file /var/FastRWeb/web.R/foo.png.R:

    # foo.png.R:
    run <- function(n=100, ...) {
      n <- as.integer(n)
      p <- WebPlot(800, 600)
      plot(rnorm(n), rnorm(n), pch=19, col=2)
      p
    }

8. I tested it with the URL: http://localhost/cgi-bin/R/foo.png?n=500

#### Security Issues

I have a feeling that if you have a "trusted machine" without user access, the steps above may not technically pose security risks (even as root); but they do not represent good security practices and *would* introduce security risks on shared servers.  For my purposes, I added to the beginning of /var/FastRWeb/code/rserve.conf:

    gid 33
    uid 33

because www-data (uid and gid 33) is the username for my apache2 instances and it seemed like a reasonable choice.  For good measure, I also changed permissions in /var/FastRWeb:

    chown www-data:www-data .
    chown -R www-data:www-data ./*

Finally, I set

    sockmod 0660
    umask 0007

based on Simon's recommendation for further security. To stop Rserve and FastRWeb:

    killall -INT Rserve

35 comments:

  1. Thank you for these instructions. I am trying to install it now on my web server.

    Now that you have used it for a couple of months, what do you think about the performance using CGI? Does it work well for you?

    C Butler

    ReplyDelete
  2. I'm very impressed. I'm close to going "live" with a new web site for a major project at Yale to be announced at the World Economic Forum in Davos later in January, and the data explorer is built with FastRWeb.

    I've learned quite a few tricks, and will propose a paper with Simon that might help folks in the end. Feel free to email me directly with a question (john.emerson@yale.edu) and I'll do my best. I will try to remember to link the new site to this blog. As you can see, I'm not a very active blogger.

    Jay

    ReplyDelete
    Replies
    1. How is the new website for the Yale project going? Is it live? Got a link?

      And the paper with Simon?

      I followed your instructions and was able to get Step 1 working successfully. However, my webhost provider does not give me root access. So I couldn't install R in step 2.

      I'm looking for a solution to this -- any recommendations?

      Delete
    2. See http://epi.yale.edu. Graphics within the Country Profiles are produced on-demand using grid graphics (as is all the HTML with the iframes).

      As for the permissions problems, I'm not sure -- it could be that if Apache is set up with certain permissions, and you are given appropriate permissions, this might be possible without root access.

      Delete
  3. Hi Dude,

    Good Post! For some strange reason some users are still using the old, unsupported JRclient which has been replaced three years ago. Thanks for sharing it.......

    Website Spider Software

    ReplyDelete
  4. Hello,

    I adapted and translated your post in French on my new blog.
    http://www.ronanm.name/fr/2012/02/fastrweb-mac/

    Thank you for this post.

    ReplyDelete
  5. I am trying to integrate R in a website and am in the process of exploring the possibilities. I read about FastRWeb and found it very interesting. Since I am also using Ubuntu I was led to your blog post by my search

    However, I am using Xampp as my developmental server, I think I would need some different steps to get it working.

    I installed FastRWeb along with the required packages and ran install.sh as instructed. I then located Rcgi and copied it to /opt/lampp/cgi-bin and renamed it R. /opt/lampp/cgi-bin the default location of Xampp's cgi-bin. I have tested that it is up and running by running the test cgi script in Xampp by entering the url "localhost/cgi-bin/test-cgi"

    I created foo.png.R as in your example and placed it in /var/FastRWeb/web.R/ (I have chmod 777 on this folder as it is a local server and only I am using my computer so security is not a big concern)

    When tested with "http://localhost/cgi-bin/R/foo.png?n=500" I got the error "Unable to connect (result=-1, socket:unknown socket error 2). "

    I tried setting "gid 33 uid 33 sockmod 0660 umask 0007" in /var/FastRWeb/code/rserve.conf without differenc. I suppose these parameters would be different for my setup but I don't know how to determine these parameters for my system.

    Your assistance will be greatly appreciated. Thanks.

    ReplyDelete
    Replies
    1. I have the same error message. Did you make any progress here?

      Delete
    2. I think this is a problem with the ownership/group/permissions of the directories in /var/FastRWeb. You might also want to stop FastRWeb and manually delete /var/FastRWeb/socket before restarting.

      Here's my setup on my Ubuntu development box:

      jay@bayesman:/var/FastRWeb$ ls -als
      total 48
      4 drwxr-xr-x 7 www-data www-data 4096 2012-04-16 14:02 .
      4 drwxr-xr-x 17 root root 4096 2011-10-02 13:05 ..
      4 drwxrwxr-x 2 www-data www-data 4096 2012-04-15 13:44 code
      4 drwxrwxr-x 2 www-data www-data 4096 2012-01-11 08:53 misc
      0 srw-rw---- 1 www-data www-data 0 2012-04-16 14:02 socket
      24 drwxrwxr-x 2 www-data www-data 20480 2012-04-16 14:02 tmp
      4 drwxrwxr-x 2 www-data www-data 4096 2011-11-07 15:45 web
      4 drwxrwxrwx 3 www-data www-data 4096 2012-03-24 17:08 web.R

      Delete
    3. Thanks Jay for your answer and for providing these detailed instructions in the first place! I finally found the solution: On an ubuntu installation your description works perfectly fine, but I was working on a CentOS copy where SElinux seems a bit stricter by default and prevents CGI scripts from opening network connections, effectively blocking the R CGI client from reaching Rserve. Here is how to change the SElinux config:

      http://bioinfogeek.blogspot.com/2012/04/fastrweb-on-centos.html

      Delete
    4. I'm taking part of this back. This helps you if you want to establish a TCP/IP connection. With unix sockets it's not helping. I have changed the user properties of the files according to yours with user apache instead of www-data on CentOS:

      4 drwxr-xr-x. 6 apache apache 4096 Apr 20 15:24 .
      4 drwxr-xr-x. 19 root root 4096 Apr 20 10:57 ..
      4 drwxr-xr-x. 3 apache apache 4096 Apr 20 15:24 code
      0 srw-rw-rw-. 1 apache apache 0 Apr 20 15:24 socket
      4 drwxr-xr-x. 3 apache apache 4096 Apr 20 13:07 tmp
      4 drwxr-xr-x. 3 apache apache 4096 Apr 19 12:05 web
      4 drwxr-xr-x. 3 apache apache 4096 Apr 20 15:08 web.R

      Unfortunately no progress, but I wanted to use TCP/IP anyway.

      Delete
    5. 1. I had the same environment as yours Ubuntu and LAMPP. I got the asme error Unable to connect (result=-1, socket:unknown socket error 2).

      2. The as pointed By Jay, I tried to start FastRweb
      /var/FastRWeb/code/start
      It gave me error
      R CMD Rserve --RS-conf /var/FastRWeb/code/rserve.conf --vanilla --no-save/usr/lib/R/bin/Rcmd: 62: exec: Rserve: not found

      helpful link : http://svn.rforge.net/FastRWeb/trunk/INSTALL

      3. The I copied the file to the directory
      sudo cp /home/amitabh/R/x86_64-pc-linux-gnu-library/3.0/Rserve/libs/Rserve /usr/lib/R/bin/Rserve

      4. The when I started FastRweb
      /var/FastRWeb/code/start
      it started successfully, and I was able to see Hello World in http://localhost/cgi-bin/R/main

      Hope this helped. Thank you.






      Delete
  6. Thanks for this. I keep getting this error


    Error in strsplit(request$raw.cookies, ";", fixed = TRUE) :
    object 'request' not found

    (from the command line) and

    Evaluation failed with error code -10

    shows up in the browser when executing the foo.png.R script (or anything for that matter). Any idea what might be the cause?

    ReplyDelete
  7. Is this a Linux installation? Were all the previous steps followed in the order described? I have to say, I have never seen this before. When you say, "from the command line" what do you mean? Aren't you trying to visit the page in your browser, like

    http://localhost/cgi-bin/R/foo.png?n=500

    ?

    ReplyDelete
  8. I got two seperate error messages, one through the browser (Evaluation code with error code -10) and the other other appeared on the command line where Rserve had been invoked.

    Anyway ,I removed the common.R script which was part of the installation, and I managed to get my own scripts working. Seems that it has something to do with the request object actually not being available.

    Thanks for this great blog post though!

    ReplyDelete
  9. Hi Jay:

    I keep getting the Error message
    Unable to connect (result=-1, socket:unknown socket error 2).

    Does this sound familiar, please?

    Thanks,
    Erin

    ReplyDelete
  10. Perhaps you didn't things up by running the script ..../FastRWeb/code/start ... and if you did, make sure there are no error messages.

    If you did do that (and if there are no error messages), then I wonder if you have a "stale" socket left over from a previous run? Make sure you do the killall as documented in my blog, and then check to see if there is an object .../FastRWeb/socket. If there is, delete it. Then start up again and try it.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. Hey Jay, hey everybody,

      I have got a similar problem as Erin had.
      My system: ubuntu 12.04 server, apache 2.2 webserver and R 3.0.1.
      If I run /var/FastRWeb/code/start in the terminal I get:
      Starting Rserve on xxx
      Loading packages...
      XML: TRUE
      Cairo: TRUE
      Matrix: TRUE
      FastRWeb: TRUE
      Rserv started in daemon mode.
      ##> SOCK_ERROR: bind error #13(access denied)

      and I have no socket-file in /var/FastRWeb.

      When I run the start-script /var/FastRWeb/code/start as root,
      everything seems to be fine.

      But I ask myself if it is usual / legal / safe to run the start-script as root?!
      Especially because on http://rforge.net/Rserve/doc.html it is said that "never ever run Rserve as root...".

      What do you think?
      Thank you very much for your help!

      Greets!
      Harry

      Delete
    3. I'm sorry, I have no idea. I've move to a Mac, myself, and am just doing development instead of a production environment -- so I use root. The only advice I have is to check the owner/permissions on every file and directory that you are working with (including apache).

      Jay

      Delete
  11. Hi again!

    Finally figured out how to start the Rserve

    However (isn't there always a however) now I'm getting an error
    ##> SOCK_ERROR: bind error #13(access denied)

    Whoops...ran start as root, all is well.
    Sorry for the trouble.

    ReplyDelete
  12. Hi Jay:

    Have you ever tried this with a Macbook air, please?

    When I try to get the http://localhost (with foo.png), I get a 'file not found on this server."

    Have you run into this please?
    Thanks,
    Erin

    ReplyDelete
  13. I'm sorry, there isn't enough information. I don't use a Mac, both others have used it successfully. It isn't clear whether (a) your web server is running -- I think it probably is, (b) you started FastRWeb -- unclear, just remember you probably have to restart every time you reboot your Mac, and (c) whether you gave the right path. It needs to be specified correctly, likely including cgi-bin/R/ as part of the path.

    ReplyDelete
  14. Hi Jay!

    It was my fault...dealing with the Rcgi.

    But it works on a Mac both on Firefox and Safari, which is good 2 know.

    Thanks,
    Erin

    ReplyDelete
  15. Hi again Jay!

    I'm running this on a "real" server (not localhost) and I keep getting "Error in file(filename, "r", encoding = encoding) : cannot open the connection" as my message. I assume that it can't find foo.png.R.

    I have sourced in into several directories, but to no avail. Have you run across this, please?

    Thanks,
    Erin

    ReplyDelete
  16. Never mind....I was using foo.png.R and not foo.png.

    What a doofus.

    Thanks,
    Erin

    ReplyDelete
  17. Jay,

    When trying to run the foo.png.R script, I get the following error:

    Evaluation failed with error code -10

    I am having a hard time finding a solution. Have you heard or seen this error? If so can you tell me what would resolve it?

    Thanks.

    ReplyDelete
  18. I did a quick Google search and it may be related to 32-bit versus 64-bit installations of FastRWeb? Make sure you started FastRWeb and that this startup was successful without errors.

    Jay

    ReplyDelete
    Replies
    1. Yes. FastRWeb is started. The issue I see after starting the package is:

      Starting Rserve on (myserver)
      Loading packages....
      XML: FALSE
      Cairo: TRUE

      Attaching package: 'Matrix'
      The following object(s) are masked from 'package:base':

      det

      Matrix: TRUE
      FastRWeb: TRUE
      Rserv started in daemon mode.

      Delete
    2. Can you do a simple "hello world" example, or the ones provided by FastRWeb, just to establish that sometime works? It isn't clear if there is a specific problem with your example, or a general problem.

      Delete
    3. Yes. Everything is working up to the last test which is the Foo script. I get the same error testing the default scripts as well, i.e., main.R

      Delete
  19. Following your instructions and using a new Ubuntu 12.04 LTS Server (32-bit ISO downloaded from Ubuntu), I was able to install FastRWeb as follows:

    sudo apt-get update && sudo apt-get upgrade
    sudo apt-get install openssh-server

    sudo apt-get install libapache2-mod-php5

    sudo apt-get install r-base

    sudo apt-get install r-cran-rserve
    sudo apt-get install libcairo2-dev
    sudo apt-get install libxt-dev

    sudo R
    install.packages("FastRWeb",dep=TRUE)
    q()

    cd /usr/local/lib/R/site-library/FastRWeb
    sudo ./install.sh
    sudo /var/FastRWeb/code/start
    sudo cp /usr/local/lib/R/site-library/FastRWeb/cgi-bin/Rcgi /usr/lib/cgi-bin/R

    Go to http://{server IP}/cgi-bin/R/example1.png .

    ReplyDelete
    Replies
    1. The following fix for broken symlinks was necessary after a server reboot before FastRWeb could be started:

      sudo rm /usr/lib/R/bin/Rserve
      sudo ln -s /usr/lib/R/site-library/Rserve/libs/Rserve /usr/lib/R/bin/Rserve
      sudo rm /usr/lib/R/bin/Rserve.dbg
      sudo ln -s /usr/lib/R/site-library/Rserve/libs/Rserve.dbg /usr/lib/R/bin/Rserve.dbg

      sudo /var/FastRWeb/code/start

      Delete
  20. This comment has been removed by the author.

    ReplyDelete
  21. This article underscores the pivotal role that data plays in today's interconnected world. From informing decision-making processes to driving innovation and efficiency, data has become the lifeblood of modern society. Thank you for the information.

    Please visit our website: https://bie.telkomuniversity.ac.id/

    ReplyDelete